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INTRODUCTION 


The  Military  Interoperable  Digital  Hospital  Testbed  (MIDHT)  is  a  five-year  program  of  research 
to  develop  a  real-world  testbed  environment  in  Southwestern  Pennsylvania.  The  purpose  is  to 
research  and  evaluate  Health  Information  Exchange  (HIE)  and  health  information  technology 
(HIT)  and  services  that  make  health  information  readily  available  to  consumers  and  providers. 
Ideally  this  will  allow  for  the  secure  transfer  of  information  between  private  sector  rural 
providers,  federal  partners  and  patients.  MIDHT  has  defined  requirements  and  solutions  to 
optimize  healthcare  resources  for  rural  communities  and  identified  lessons  learned  and  best 
practices  that  benefit  both  the  global  Military  Health  System  (MHS)  environment  and 
stakeholders  in  the  region.  The  Department  of  Defense  (DoD)  and  Conemaugh  Memorial 
Medical  Center  (CMMC)  have  common  requirements  for  HIE,  connecting  disparate  systems  and 
providers  and  enabling  secure  provider-provider  and  provider-consumer  e-communications. 
Minimal  evidence  is  available  on  what  business,  clinical  and  technical  solutions  can  be  used  to 
overcome  the  lack  of  specialists,  infrastructure  and  geographical  barriers  associated  with  the 
delivery  of  care  in  rural  communities. 


BODY 


Arm  1:  Longitudinal  Study  for  Use  of  Interoperable  Accessible  Health 
Information  Exchange  Services  and  Technologies  in  Rural 
Communities  (A  -  15835.2,  A  -  16192.1). 

This  arm  focused  on  ways  a  rural  environment  can  capitalize  on  the  use  of  health  information 
network  (HIM)  services  and  technologies  to  promote  interoperability  between  disparate  entities 
such  as  TRICARE  providers,  private  sector  health  systems,  and  DoD  facilities.  MIDHT 
investigated  attitudes,  usability,  and  effectiveness  of  HIN  services  by  rural  providers,  including 
the  effect  of  the  use  of  HIE  tools  by  provider  groups,  TRICARE  providers,  and  three  CHS 
facilities  on  their  business  practices  and  process  flows.  Research  initiatives  focused  on  the 
impact  of  an  electronic  health  record  (EHR)  implementation  using  instruments  utilized  in  year  1 . 
Additionally,  research  initiatives  evaluated  the  ability  to  electronically  access  digital  radiology 
images  and  how  this  system-wide  functionality  affected  the  delivery  of  patient  care  within  a  rural 
health  care  system,  to  include  an  analysis  on  provider  productivity,  throughput,  duplicative 
testing  and  continuity  of  care.  Finally,  an  assessment  of  the  volume  of  cases  that  Conemaugh 
physicians  have  with  the  Social  Security  Administration  (SSA)  regarding  veteran/military 
disability  claims  was  completed  through  a  provider  satisfaction  survey  regarding  the  existing 
SSA  process. 
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Subtask  1.1  Assess  changes  in  provider  workflows  and  efficiency  resulting  from  the 
implementation  of  an  ambulatory  electronic  medical  record. 

Study  A-1 5835.2 

Respective  protocol  received  Conemaugh  administrative,  scientific  and  IRB  approvals  during  the 
third  quarter  of  Funding  Year  1.  USAMRMC  IRB  approval  was  granted  on  May  18,  2010, 
allowing  Conemaugh  to  begin  study  implementation.  Distribution  of  EHR  surveys  to 
staff/physicians  and  comprehensive  workflow  shadowing  of  staff/physicians  at  both  locations 
began  in  June  2010  as  the  EHR  stabilization  period  (3  months)  ended  per  protocol  design. 

After  the  implementation  of  an  EHR  and  a  three-month  stabilization  period,  19  providers 
between  the  Portage  and  NORCAM  ambulatory  facilities  (Table  1)  were  each  shadowed  for  a 
continuous  four-hour  time  period.  Researchers  used  the  Time  and  Motion  Study  Tool: 
Ambulatory  Practice  (TMS-AP)  developed  by  Partners  Healthcare,  in  order  to  be  consistent  with 
previous  completed  work.  The  new  data  set  (“POST”)  from  2010  allowed  for  a  statistical 
comparison  to  the  “PRE”  data  set  collected  in  2008. 


Location 

Subject 

Date 

Portage 

Clerical 

6/14/2010 

Portage 

Clinical 

6/14/2010 

Portage 

Physician 

6/14/2010 

NORCAM 

Clinical 

6/21/2010 

NORCAM 

Physician 

6/21/2010 

NORCAM 

Clerical 

6/21/2010 

Portage 

Clinical 

6/28/2010 

Portage 

Clerical 

6/28/2010 

Portage 

Physician 

6/28/2010 

NORCAM 

Physician 

6/30/2010 

NORCAM 

Clerical 

6/30/2010 

NORCAM 

Clinical 

6/30/2010 

Portage 

Physician 

7/7/2010 

Portage 

Office  Manager 

7/7/2010 

Portage 

Clerical 

7/7/2010 

NORCAM 

Office  Manager 

7/19/2010 

NORCAM 

Clerical 

7/19/2010 

Portage 

Clinical 

8/13/2010 

Portage 

Physician's  Assistant 

11/4/2010 

Table  1.  EHR  POST  Observations  Schedule 

The  following  section  describes  the  statistical  analysis  completed  when  comparing  the  PRE  and 
POST  workflow  data  sets. 
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[Mann-Whitney  Test  (MWT)  with  Alpha  =  0.05  was  used  for  all  significance  testing.  Note,  MWT 
utilizes  ranks  and  not  means. \ 

The  variable,  percent  of  time  period  total  time,  was  calculated  independently  for  each  time  period 
(PRE  and  POST)  using  the  same  procedure;  namely,  the  time  for  each  observed  activity  was 
normalized  by  dividing  it  by  the  total  time  of  the  period  in  which  it  was  recorded  and  then 
multiplying  the  dividend  by  100.  This  mitigates  the  difference  in  total  time  between  time  periods 
and  allows  for  more  accurate,  interpretable,  and  directly  comparable  results  between  time 
periods.  For  readability,  the  variable,  percent  of  time  period  total  time,  will  henceforth  be 
denoted  as  percent  time. 


Aggregated: 

Location 

paper  or 

electronic 

Time  Period 

N 

d(N) 

Sum 

d(Sum) 

Mean 

d(Mean) 

paper 

PRE 

510 

-297 

11.837 

-7.821 

0.02321 

-0.004357 

NORCAM 

post 

213 

4.016 

0.01885 

ELECTRONIC 

PRE 

149 

751 

3.534 

11.379 

0.02372 

-0.007146 

post 

900 

14.912 

0.01657 

paper 

PRE 

540 

-237 

"■l3'.244' 

-10.248 

0.02453 

-0.014639 

Portage 

post 

303 

2.996 

0.00989 

ELECTRONIC 

PRE 

336 

737 

7.324 

7.920 

0.02180 

-0.007590 

post 

1073 

15.244 

0.01421 

Table  2.  Paper  vs.  Electronic  Usage 


Note:  the  d()  nomenclature  denotes  change-in  that  quantity,  (POST  -  PRE) 
d(N)  =  change  in  the  number  of  observed  activities 

d(Sum)  =  change  in  the  sum  of  percent  time  of  the  activities  contained  in 

the  grouping  indicated 

d(Mean)  =  change  in  the  average  percent  time  elapsed  per  observed  activity 

A  significant  change  (POST  -  PRE)  exists  for  both  NORCAM  and  Portage  for  activities 
aggregated  into  categories  of  paper  and  electronic.  Also  of  note  are  the  magnitude  and  direction 
of  change  demonstrated  by  d(N)  and  d(Sum): 


Change-in, 

(variable 

description) 

variable 

Paper, 

(aggregated) 

ELECTRONIC, 

(aggregated) 

NORCAM 

Portage 

NORCAM 

Portage 

Activity  count 

d(N) 

58%  reduction 

44%  reduction 

5  fold  increase 

2  fold  increase 

Percent  time  Sum 

d(Sum) 

8%  reduction 

10%  reduction 

11%  increase 

8%  increase 

Table  3.  Paper  vs.  Electronic  Usage 
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Time  and  Motion  Conciusion 


A  significant  negative  change  (post  -  pre)  on  the  variable,  percent  time,  was  found  for  various 
processes  at  Portage  and  NORCAM.  The  results  for  NORCAM  suggest  that  productivity  (on  a 
time  basis)  improved  for  medication  orders  and  renewals,  receiving  lab/test  results,  medical 
records  management,  phone  calls  with  patients,  scheduling  patient  appointments  and  revenue 
cycle.  The  results  for  Portage  suggest  that  productivity  (on  a  time  basis)  improved  for 
medication  orders  and  renewals,  writing  orders  and  scheduling  tests,  receiving  lab/test  results, 
medical  records  management,  phone  calls  with  patients,  scheduling  patient  appointments  and 
revenue  cycle.  Investigators  conclude  that  the  implementation  of  an  electronic  health  record 
directly  improved  medication  orders  and  renewals,  receiving  lab/test  results,  and  medical  records 
management;  the  electronic  health  record  did  not  improve  productivity  associated  with  clinical 
notes,  transcription  and  dictation.  Furukawa'  found  that  electronic  medical  records  have  a  mixed 
association  with  efficiency  and  productivity  during  office  visits. 

In  reference  to  Table  5,  N  is  defined  as  the  aggregate  number  of  distinct  activities  performed 
during  the  Time  &  Motion  observations.  The  unit  for  the  “Sum”  column  is  hours.  The  negative 
d(Mean)  can  be  interpreted  as  a  decrease  in  the  average  of  percent  time  required  for  an  activity, 
which  implies  increased  productivity. 


Process 

ID  Number 

Medication  Orders/Renewals 

1 

Write  Dx  Orders/Scheduling  Tests  &  Referrals 

2 

Receiving  Lab/Test  Results 

3 

Medical  Records  Management 

4 

Clinical  Notes/Transcription/Dictation 

5 

Phone  -  Patient 

6 

Scheduling  Patients  in  Office 

7 

Talking  -  Colleague/Walking  Inside 

8 

Revenue  Cycle 

9 

Remaining  Activities  Not  Under  Analysis 

0 

Table  4.  Process  Description 
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Location 

MIDHT 

Process 

Time 

Period 

N 

d(N) 

Sum 

d(Sum) 

Mean 

d(Mean) 

0 

PRE 

441 

16.914 

0.750 

0.03970 

-0.01933 

post 

17.664 

0.02037 

1 

PRE 

34 

1.011 

0.161 

0.02807 

-0.01133 

post 

70 

0.01674 

2 

PRE 

38 

35 

0.695 

0.02651 

-0.00319 

N 

post 

73 

1.703 

0.02332 

3 

PRE 

34 

-18 

0.749 

-0.584 

0.02203 

-0.01174 

0 

post 

16 

0.165 

0.01029 

4 

PRE 

201 

320 

2.814 

0.02747 

-0.01147 

R 

post 

521 

0.01600 

c 

5 

PRE 

-155 

5.746 

-3.440 

0.02262 

0.00067 

post 

2.306 

0.02330 

A 

6 

PRE 

81 

29 

2.373 

-0.005 

0.02930 

-0.00777 

M 

post 

110 

2.368 

0.02153 

7 

PRE 

67 

147 

1.398 

1.659 

0.02086 

-0.00657 

post 

214 

3.057 

0.01428 

8 

PRE 

232 

-20 

3.054 

-0.140 

0.01316 

0.00058 

post 

212 

2.914 

0.01375 

9 

PRE 

60 

28 

1.240 

0.237 

0.02067 

-0.00388 

post 

88 

1.477 

0.01679 

0 

PRE 

770 

687 

29.032 

2.023 

0.03770 

-0.01639 

post 

1457 

31.056 

0.02131 

1 

PRE 

74 

-48 

2.358 

-2.006 

0.03186 

-0.01833 

post 

26 

0.01353 

2 

PRE 

104 

i; 

-0.629 

0.02266 

-0.00681 

P 

post 

109 

1.728 

0.01585 

n 

3 

PRE 

42 

_9 

0.900 

-0.209 

0.02143 

-0.00417 

w 

post 

0.691 

0.01726 

r 

4 

PRE 

313 

171 

9.066 

-3.537 

0.02896 

-0.01754 

f 

post 

484 

5.529 

0.01142 

i 

5 

PRE 

30 

5.419 

0.428 

0.02258 

-0.00092 

a 

post 

wm 

5.847 

0.02166 

6 

PRE 

137 

109 

5.154 

-2.173 

0.03762 

-0.02550 

9 

post 

246 

2.981 

0.01212 

e 

7 

PRE 

32 

231 

0.702 

2.108 

0.02194 

-0.01126 

post 

263 

2.810 

0.01068 

8 

PRE 

351 

201 

5.376 

1.035 

0.01532 

-0.00370 

post 

552 

6.411 

0.01161 

9 

PRE 

9 

97 

0.623 

0.812 

0.06917 

-0.05563 

post 

106 

1.435 

0.01353 

Table  5.  Time  &  Motion  Summary 


Survey  opportunities  were  made  available  to  providers  so  that  qualitative  assessment  of  user 
satisfaetion  of  the  Allscripts  EHR  could  be  undertaken.  The  survey  used  was  retrieved  from  the 
Agency  for  Healthcare  Research  and  Quality  (AHRQ)  survey  compendium  and  was  modified  for 
project  use.  An  attempt  was  made  to  determine  if  satisfaction  changed  over  time  (June  2010, 
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November  2010  and  May  2011)  but  a  low  sample  size  prevented  analysis.  Below  are  descriptive 
results  of  some  important  questions  (n=26): 


The  EHR  provides  me  with  all  the  information  I  need  to  take  care  of  the 

patient 


I  Never 
I  Sometimes 
I  Occasionaly 
I  Most  of  the  time 
I  Always 


Figure  1.  EHR  Utility. 


How  satisfied  are  you  with  the  EHR  system? 


Dissatisfied  Somewhat  Satisfied 


Very  Dissatisfied 


Neutral 


Very  Satisfied 


Other 


I  Physician 
I  Office  Manager 
I  Nurse 


■■  Clerical 


Figure  2.  Satisfaction  with  EHR. 
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The  system  lets  me  quickly  find  the  information  I  need. 


Figure  3:  EHR  Satisfaction  Results 

Survey  Conclusion 

Overall,  the  staff  at  Portage  and  NORCAM  indicated  an  ambivalence  regarding  satisfaction  of 
the  EHR  implementation  with  almost  one  third  stating  a  neutral  satisfaction.  When  looking  at 
position  type,  no  apparent  trends  were  noted  (Note:  “Other”  most  likely  includes  some 
physicians).  These  satisfaction  results  are  much  lower  than  those  reported  by  DesRoches  et  al 
regarding  a  2008  national  survey  of  physicians  on  EHRs.  Only  34%  of  CMMC  staff  was 
satisfied  whereas  90%  of  physicians  surveyed  nationally  were  satisfied.  Despite  the  finding  of 
neutral  satisfaction,  descriptive  analysis  of  certain  questions  indicates  a  positive  perception  of  the 
EHR.  Aggregating  the  answer  choices  for  question  twenty  (q0020)  as  either  difficult  or  easy 
shows  that  a  majority  of  respondents  indicated  the  following  activities  were  easier  when  using 
the  EHR  as  compared  to  previous  routines: 

•  Documenting  allergies 

•  Documenting  CPT  and  ICD-9  codes  for  billing  purposes 

•  Keeping  problem  lists  updated 

•  Reviewing  laboratory  and  radiology  results 

•  Writing  and  renewing  prescriptions 

•  Monitoring  medication  safety  during  prescribing 

•  Communicating  referral  information  to  specialists 
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Furthermore,  approximately  50%  of  respondents  agreed  that  the  EHR  enabled  them  to 
accomplish  tasks  more  quickly,  enhanced  job  effectiveness  and  made  it  easier  to  do  their  job. 
Whereas  45%  of  respondents  state  they  work  longer  hours  to  see  the  same  number  of  patients. 
Also,  no  consensus  that  the  EHR  caused  disruptions  to  their  workflow  was  found. 

Written  comments  most  often  cited  suggest  that  staff  appreciated  not  having  to  pull  and  refde 
paper  charts,  access  to  hospital  information  (e.g.  lab/rad  results)  was  much  easier  (direct  interface 
to  Allscripts),  and  the  benefits  of  ePrescribing.  On  the  negative  side,  staff  members  were 
frustrated  with  the  multiple  screens  and  not  being  able  to  quickly  locate  information,  creating 
clinical  notes  was  too  cumbersome,  determining  which  tests  were  ordered  was  more  difficult  and 
more  training  was  needed. 

Although  the  significance  of  the  following  statistical  analysis  cannot  be  interpreted  as  definitive 
due  to  violations  resulting  from  the  sample  size,  the  results  do  generally  support  the  conclusions 
drawn  from  the  descriptive  analysis  above. 

Statistical  Analysis  of  EHR  Survey  Results  (n=26): 


Crosstabs  were  performed  over  questions  1-19  (rows)  on  questions  22,  27,  28,  29,  30  and  31 
(columns).  Chi-square  was  used  to  test  for  significance  between  the  columns  over  the  rows. 
Unfortunately,  the  sample  size  produced  untenable  cell  counts  and  significance  testing  could  not 
be  performed.  Furthermore,  a  crosstab  of  question  30  and  3 1  did  not  produce  a  significant  result 
using  Chi-square;  nor  did  question  22  with  30.  Analyzed  independently,  questions  20  and  21 
yielded  slightly  positive  responses,  overall.  The  low  sample  size  overall  and  decline  in  responses 
over  time  prevented  a  longitudinal  analysis  of  survey  data  as  designed.  Please  refer  to  Appendix 
1  for  a  copy  of  the  survey  results. 
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S04_3_q0022 

S04_8_q0027 

S04_9_q0028 

S04_10_q0029 

S04_11_q0030 

S04_12_q0031 

S02_q0001 

0.274 

0.108 

0.295 

0.008 

0.200 

0.085 

S02_q0002 

0.063 

0.568 

0.615 

0.032 

0.511 

0.005 

S02_q0003 

0.020 

0.644 

0.390 

0.188 

0.267 

0.037 

S02_q0004 

0.018 

0.433 

0.356 

0.728 

0.080 

0.102 

S02_q0005 

0.011 

0.719 

0.347 

0.739 

0.817 

0.012 

S02_q0006 

0.225  (0.378) 

0.431  (0.695) 

0.018 

0.743 

0.027  (0.073) 

0.053 

S02_q0007 

0.255 

0.988 

0.398 

0.379 

0.203 

0.013 

S02_q0008 

0.320 

0.486 

0.378 

0.389 

0.037 

0.094 

S02_q0009 

0.819 

0.793 

0.727 

0.384 

0.039 

0.322 

S02_q0010 

0.206 

0.930 

0.059 

0.837 

0.022 

0.049 

S02_q0011 

0.161 

0.550 

0.223 

0.753 

0.759 

0.567 

S02_q0012 

0.151 

0.253 

0.210 

0.012 

0.648 

0.330 

S03_1_q0013 

0.030 

0.136 

0.511 

0.185 

0.626 

0.169 

S03_2_q0014 

0.070 

0.574 

0.338 

0.255 

0.763 

0.592 

S03_3_q0015 

0.362 

0.113 

0.957 

0.424 

0.917 

0.920 

S03_4_q0016 

0.332 

0.756 

0.966 

0.238 

0.799 

0.691 

S03_5_q0017 

0.041 

0.105 

0.556 

0.018 

0.317 

0.093 

S03_6_q0018 

0.023 

0.134 

0.022 

0.255 

0.378 

0.160 

S03_7_q0019 

0.229 

0.596 

0.055 

0.753 

0.039 

0.133 

NOTES:  1 )  Values  in  this  table  represent  the  Pearson  Chi-square  result  except  where  indicated  below. 

2)  ( )  indicate  the  asymptotic  significance  as  calculated  by  the  Fisher's  Exact  Test  for  a  2x2  matrix 

3)  italics  indicate  that  fewer  than  25%  of  the  cells  have  an  expected  count  <  5 

4)  All  results  (except  those  in  italics)  result  from  a  matrix  that  has  at  least  33%  of  the  cells  with  an 
expected  count  <  5.  As  such,  these  results  should  not  be  used  to  make  concrete  inferences  as  to 
significance. 

Table  6.  Chi-Square  p-values  for  Crosstabs  of  Survey  Questions 
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Crosstab 


S04_3_ 

_q0022 

No 

Yes 

Total 

No 

S04  11 

Count 

%  within 

S04  3  q0022 

5a 

71.4% 

13a 

72.2% 

18 

72.0% 

_q0030 

Yes 

Count 

%  within 

S04  3  q0022 

2a 

28.6% 

5a 

27.8% 

7 

28.0% 

Total 

Count 

%  within 
S04_3_q0022 

7 

100.0% 

18 

100.0% 

25 

100.0% 

Each  subscript  letter  denotes  a  subset  of  S04_3_q0022  categories 
whose  column  proportions  do  not  differ  significantly  from  each  other 
at  the  .05  level. 


Chi-Square 


Tests 


Asymp. 

Exact 

Exact 

Sig. 

Sig. 

Sig. 

Value 

df 

(2-sided) 

(2-sided) 

(1 -sided) 

Pearson  Chi- 
Square 

.002^ 

1 

.968 

Continuity 

Correction‘s 

.000 

1 

1.000 

Fisher's  Exact 
Test 

1.000 

.663 

N  of  Valid 

25 

Cases 

1  a.  1  cell  (25.0%)  has  an  expected  count  less  than  5.  The  1 

minimum  expected  count  is  1.96. 

b.  Computed  only  for  a  2x2  table 

Table  7.  Crosstab  Results  and  Chi-Square  p-value  for  Survey  Questions  1 1  and  22 


Crosstab 


1  S04  3  q0022  | 

Total 

No 

Yes 

Count 

Dissatisfied  o/„  ^jthjn 

S04  3  q0022 

4a 

57.1% 

2b 

11.1% 

6 

24.0% 

VERY 

Dissatisfied  within 

S04  3  q0022 

la 

14.3% 

2a 

11.1% 

3 

12.0% 

S04  12  .  , 

q0031  neutral  within 

S04  3  q0022 

2a 

28.6% 

5a 

27.8% 

7 

28.0% 

.  ,  Count 

somewhat 

satisfied  within 

S04  3  q0022 

Oa 

.0% 

8b 

44.4% 

8 

32.0% 

VERY 

satisfied  within 

S04  3  q0022 

Oa 

.0% 

1a 

5.6% 

1 

4.0% 

Count 

Total  %  within 

S04_3_q0022 

7 

100.0% 

18 

100.0% 

25 

100.0% 

Each  subscript  letter  denotes  a  subset  of  S04_3_q0022  categories  whose 
column  proportions  do  not  differ  significantly  from  each  other  at  the  .05 

level. 


Chi-Square  Tests 


Value 

df 

Asymp.  Sig. 

(2-sided) 

Pearson 

Chi- 

Square 

7.993^ 

4 

.092 

N  of 
Valid 
Cases 

25 

a.  8  cells  (80.0%)  have  expected 
count  less  than  5.  The  minimum 
expected  count  is  .28. 


Table  8.  Crosstab  Results  and  Chi-Square  p-value  for  Survey  Questions  12  and  22 
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S04_11_q0030  *  S04_12_q0031  Crosstabulation 


S04_12_q0031 

Total 

Dissatisfied 

VERY 

Dissatisfied 

neutral 

somewhat 

satisfied 

VERY 

satisfied 

S04_1 1_q0030  No  Count 

%  within  S04  12  q0031 

2a 

33.3% 

2a 

66.7% 

7a 

87.5% 

7a 

87.5% 

1a 

100.0% 

19 

73.1% 

Yes  Count 

%  within  S04  12  q0031 

4a 

66.7% 

la 

33.3% 

la 

12.5% 

la 

12.5% 

Oa 

.0% 

7 

26.9% 

Total  Count 

%  within  S04_12_q0031 

6 

100.0% 

3 

100.0% 

8 

100.0% 

8 

100.0% 

1 

100.0% 

26 

100.0% 

Each  subscript  letter  denotes  a  subset  of  S04_12_q0031  categories  whose  column  proportions  do  not  differ  significantly  from  each 
other  at  the  .05  level. 


Chi-Square  Tests 


Value 

df 

Asymp.  Sig. 
(2-sided) 

Pearson  Chi-Square 

N  of  Valid  Cases 

6.940" 

26 

4 

.139 

a.  8  cells  (80.0%)  have  expected  count  less  than  5. 
The  minimum  expected  count  is  .27. 


Table  9.  Crosstab  Results  and  Chi-Square  p-value  for  Survey  Questions  30  and  31 

Statistieal  Analysis  of  Produetivitv  Data  (n  =  6  providers  (eontrol).  5  providers  (EHR)): 

Initial  statistical  analysis  obviated  the  necessity  for  additional  data  points  to  achieve  sufficient 
symmetry  and  homogeneity  to  produce  an  analyzable  dataset.  A  protocol  modification  was 
necessary  to  further  clarify  the  time  period  to  be  used  (January  2007  -  February  2010)  for 
analysis,  which  was  approved  by  Conemaugh  IRB  on  December  9,  201 1. 

Quarterly  data  was  obtained  for  years  2007  through  2011.  Both  a  MANOVA  and  RM- 
(m)ANOVA  were  attempted;  however,  underlying  assumptions  for  those  tests  were  violated.  The 
data  was  reorganized  and  recast  to  conform  to  a  paired-t  and  a  standard  independent  t-test.  Since 
Year  2007  only  contained  data  for  the  following  variables.  Charges,  Units,  Encounters,  and 
Office  Hours  Worked,  it  could  not  be  included  in  the  paired-t  or  independent  t-test  derived  from 
the  paired-t  setup.  Furthermore,  the  calendar  quarters  for  all  years  were  not  consistent  due  to  the 
implementation.  To  minimize  the  impact  of  potential  periodicity,  the  data  pairing  for  the  paired-t 
used  2009,  2010,  and  2011  data.  The  paired-t  test  was  used  to  investigate  the  within-group 
change  between  the  post  and  pre  time  periods.  Due  to  the  testing  of  multiple  dependent  variables, 
the  test-wise  alpha  was  adjusted  so  that  the  overall  alpha  remained  0.05.  The  following 
calculated  variables  were  produced  so  as  to  provide  for  a  more  direct  comparison  between  the 
control  and  intervention  groups:  Charge/Unit,  Encounters/Hr,  and  Total  RVUs  per 
Encounters/Hour.  The  difference  (post-pre)  was  calculated  and  a  standard  t-test  was  applied  to 
test  if  the  mean  difference  was  statistically  significant  between  groups.  No  statistical 
significance  was  found. 
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Despite  the  finding  of  no  statistieal  signifieanee  between  the  control  and  intervention  group,  the 
change  (post  -  pre)  in  the  mean  difference  of  encounters  per  hour  improved  for  the  intervention 
group  and  not  the  control  group  (0.28  vs  -0.086).  A  similar  improvement  was  also  noted  for 
Total  RVUs  (546  vs  316).  In  other  words,  EHR  users  likely  realized  an  increase  in  patients  per 
hour  and  productivity  when  compared  to  the  paper-based  physician  offices. 

Study  Closure: 

Subject  study  was  closed  with  Conemaugh  IRB  on  April  2,  2012.  Final  report  and  supporting 
documents  were  delivered  to  USAMRMC  ORP  HRPO  and  found  to  be  acceptable  on  April  19, 
2012  (refer  to  Appendix  2). 

Subtask  1.2  Enhance  the  service-based  HIE  infrastructure  and  services  to  support  further 
exchange  of  digital  medical  imaging  information  in  a  rural  setting. 

Extending  the  McKesson  PACs,  Radiology  Information  System  (RIS)  and  Dolbey  Digital 
Dictation  system  used  at  Memorial  Medical  Center  to  Miners  and  Meyersdale  Medical 
Centers  allowed  Conemaugh  to  achieve  consistency  across  the  health  system  when  it  comes  to 
radiology  imaging,  report  management,  and  access.  This  was  of  great  value  to  the  physicians  and 
patients  of  the  health  system  allowing  a  reduction  in  the  redundancy  of  diagnostic  testing  and 
ultimately  provides  better  care  to  our  patients. 

Memorial  Medical  Center’s  MIS  Department  worked  closely  with  resources  from  McKesson’s 
MRM  and  HMl  divisions.  Relevant  system  design  and  table  builds  were  identified.  Workflow 
processes  among  the  three  entities  were  reviewed  and  altered  to  accommodate  the  new  integrated 
process.  Workflow  and  policies  for  establishing  Miners  and  Meyersdale  on  the  Memorial  PACS 
system  were  designed.  Radiologists  and  technologists  reading  and  workstations  were  purchased, 
configured,  and  installed  at  both  facilities,  and  the  set  up  of  off-hour  image  transmission  was 
completed.  In  addition,  Vidar  film  digitizers,  computerized  radiology  (CR)  devices,  document 
scanners,  and  Dolbey  dictation  equipment  and  software  licenses  were  obtained  and  installed  at 
both  facilities.  McKesson  PACS  software  licenses  for  MyMC  and  MiMC  facilities  were 
purchased,  and  staff  from  Miners  and  Meyersdale  were  trained  by  Memorial  Radiology  and 
Management  Information  Systems  Departments  on  the  use  of  PACS.  Workgroups  were  built 
into  PACS  to  accommodate  the  workflow  for  Miners  and  Meyersdale  and  the  networking  group 
provided  needed  cabling  and  networking  services  to  provide  workstation  and  modality  network 
connectivity  and  access  from  both  locations. 

On  July  1,  2009,  PACS  went  live  at  Miners  and  Meyersdale  for  image  transfer  only.  Radiology 
staff  at  both  locations  continued  to  be  trained  on  RIS  functionality.  Tables  were  identified  to  be 
scripted  for  the  test  environment  and  cross-reference  exam  master,  charge  master  was  added  to 
the  spread  sheet  to  be  used  for  scripting  in  large  tables.  McKesson  upgraded  the  test  server  and 
retested  the  application.  Reports  that  were  added  by  Miners  and  Meyersdale  were  tested,  but 
were  found  to  have  an  incorrect  footer.  Application  testing/HRM  HIM  integration  testing 
occurred,  with  status  changes  from  HMl  to  HRM  not  working.  This  was  later  corrected.  The 
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following  HRM  tables  were  made  available  in  Live:  Facilities,  Locations,  Modalities,  Work  list. 
Film  Libraries,  Film  jacket  types,  and  locations  and  Pt  Class.  The  following  tables  were  edited  in 
Live  to  be  Enterprise  wide:  Admin  Route,  Pt  Condition,  and  Normals.  EMPLADT/lntegration 
testing  was  completed. 

Productive  use  via  the  McKesson  Care  Portal  was  implemented  on  January  19,  2010  and  post- 
live  support  was  provided  to  Miners  and  Meyersdale.  Weekly  calls  continued  between  Miners, 
Meyersdale,  and  CMMC  to  identify  post-live  issues.  McKesson  assisted  with  vendor  identified 
issues.  Calls  continued  weekly  to  validate  processes  and  resolve  issues.  McKesson  continued  to 
be  engaged  to  assist  with  patient  merge  issues. 

Subtask  1.3  Research  and  evaluate  the  ability  to  electronically  exchange  digital  images  and 
how  this  functionality  will  affect  the  delivery  of  patient  care  within  a  rural  health 
care  system,  to  include  an  analysis  on  provider  productivity,  throughput, 
duplicative  testing  and  continuity  of  care. 


Study  16192.1 


The  Initiate  Enterprise  Master  Patient  Index  (EMPI)  software  was  used  to  correlate  and  match 
different  medical  record  numbers  across  facilities  for  the  same  patient.  Due  to  this  procedure, 
duplicate  tests  within  the  same  facility  were  identified  by  the  study  team  with  a  high  degree  of 
confidence.  Significance  testing  for  hypothesis  #1  was  completed  with  the  following  results  for 
the  count  of  tests  originating  at  Miners/Meyersdale  and  duplicated  at  Memorial  (CPT  71010, 
71020,  70450)  by  time  period  is  presented  below: 


PRE  (baseline) 

POST 

Phase  1  '  Phase  2 

_ 1 _ 

2008 

2009  \  2010 

July  through 

December 

January  through 
June 

1 

July  through  |  January  through 

December  i  June 

1 

Table  10:  Study  Time  Period 

A  Chi-Square  test  of  independence  applied  to  either  Days  between  date  of  service  (DOS)  or 
aggregated  Days  between  DOS  over  time  period  did  not  yield  a  significant  result  yet  a  7% 
reduction  in  duplicate  imaging  is  noted.  No  statistical  significance  was  found  by  CPT  code. 
Radiology  volume  data  by  CPT  code  for  Miners  and  Meyersdale  was  homogeneous  between  the 
pre  and  post  data  sets.  The  results  were  more  favorable  than  what  You^  discovered  in  a  similar 
duplicate  imaging  study  in  Canada.  You  found  a  0.1%  reduction  in  chest  x-rays  and  0.2% 
reduction  in  CT  scans  of  the  head  whereas  CMMC  researchers  found  a  2.9%  reduction  in  chest 
x-rays  and  a  26.8%  reduction  in  CT  scans  of  the  head  for  the  study  time  period. 
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Count 


Time  Period 

Ipre 

■post 


Remote  Site  2 


Remote  Site  1 


HOSPITAL  OFF-site 


2  3  4  saea? 

Days  bfw  DOS,  aggregated 


Figure  4.  Duplicate  Tests  by  Hospital  and  Days  Between  Date  of  Service. 


Time  Period 

■pre 

■post 


CT  scan,  head  Chest  X-ray  Chest  X-ray 
(s  ingle  view)  (double  view) 


Test  (Image)  Type 


Figure  5.  Duplicate  Tests  by  Image  Type. 
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Aggregated 
Days  b/w  DOS 

CPT 

Code 

pre 

POST 

Total 

Change, 
(POST -PRE) 

All- 

study 

0&  1 

specific 

203 

213 

416 

10 

2 

42 

30 

72 

-12 

3 

25 

16 

41 

-9 

4 

21 

11 

32 

-10 

5&6&7 

33 

31 

64 

-2 

Column  Totals: 

324 

301 

625 

-7% 

Table  11.  Raw  Data  Counts. 

PACS  User  Reports 

A  randomized  proportionate  sample  of  PACS  users  on  the  CMMC  medical  staff  were  identified 
to  collect  data  on  their  viewing  of  images  that  originated  from  Miners  and/or  Meyersdale 
facilities  during  October  -  December  2009  and  April  -  June  2010.  Management  Information 
Systems  provided  reports  to  the  study  team,  38%  (27/70)  of  physicians  had  applicable  data. 
When  comparing  POST  Phase  1  &  2  data,  count  is  very  similar  so  they  were  consolidated. 

As  depicted  below,  the  most  active  users  of  the  PACS  system  in  terms  of  viewing  images 
originating  from  Miners  and  Meyersdale  are  Emergency  Medicine  and  Trauma  physicians.  This 
result  is  expected  as  Conemaugh  Memorial  Medical  Center  (CMMC)  is  a  tertiary  care  hospital 
with  a  Level  I  trauma  center.  The  next  level  includes  Otolaryngology,  Urology,  and  Pulmonary. 
A  third  level  includes  General  Surgery,  Orthopedics  and  Neurosurgery. 


Specialty 

Studies  From 

Remote  Sites 

Viewed  \ia 

PACS 

Emergency  Medicine 

65 

Trauma 

51 

Otolaryngology' 

35 

Urology' 

34 

Pulmonology' 

31 

General  Surgery' 

28 

Orthopedics 

27 

Neurosurgery' 

26 

Internal  Medicine 

18 

Vascular 

10 

OB/GYN 

5 

Nephrology' 

2 
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Table  12.  PACS  Usage  by  Specialty. 


Physician  Surveys 


The  qualitative  surveys  collected  that  indicated  that  the  physician  never  used  PACS  to  access 
images  from  Miners  and  Meyersdale  were  removed  from  the  dataset  before  analysis.  The 
remaining  responses  (n=55)  formed  the  dataset  of  analysis.  Although  not  statistically  significant 
(alpha=0.05),  the  bar  chart  below  clearly  shows  that  those  respondents  who  indicated  having 
previous  experience  with  PACS  used  the  system  more  than  those  who  had  not  had  previous 
PACS  experience. 

Investigators  decided  to  focus  on  the  following  questions: 


Bar  Chart 


15.  Have  you 
had  previous 
experience 
with 

RADIOLOGY 
PACS  outside 
of 

Conemaugh? 

■  Yes 
UNo 


Frequently  Sometimes  Seldom 

3.  How  often  do  you  access  electronic  images  for 
radiology  exams  performed  at  Miners  and  Meyersdale 
via  Care  Portal  and/or  standalone  PACS  viewers  (Ml 
View  or  HRS-D)? 


Figure  6:  Frequency  of  PACS  Usage. 


The  results  suggest  that  the  PACS  implementation  at  rural  hospitals  (Miners  and  Meyersdale) 
had  a  positive  impact  on  providers  and  patients.  A  majority  of  physieians  (87%)  believed  that 
image  access  has  improved  productivity  whereas  81%  of  physicians  believed  that  immediate 
PACS  image  aeeess  has  improved  physieians’  ability  to  make  decisions  regarding  patient  eare. 
Furthermore,  70%  of  physicians  agree  that  immediate  PACS  image  aeeess  has  redueed  the 
number  of  exams  reordered  beeause  images  taken  previously  were  not  known  about  or  not 
available  in  a  timely  manner. 


Tablets.  Chi-Square  Tests 


Value 

df 

Asymp.  Sig. 
(2-sided) 

Pearson  Chi- 
Square 

4.944a 

2 

0.084 

Likelihood  Ratio 

5.002 

2 

.082 

Linear-by-Linear 

Association 

3.270 

1 

.071 

N  of  Valid  Cases 

55 

a.  0  cells  (.0%)  have  expected  count  less  than  5. 
The  minimum  expected  count  is  6.36. 


Page  20  of  99 


Productivity:7a.  My  productivity  has  improved  because  of  image  access. 
(Piease  check  ONLY  one  response) 


30 


Moderateily  Degree  Stroinglly  Agree 


Stnoinglly  Disagree 


Mode  rate  fy  Agree 


Mot  Applies  b  le 


I  Seldomi 
I  SorTvetimes 
I  Frequently 


Figure  7.  Productivity  Improvement. 


Decision  MakingrSa.  Immediate  PACS  image  access  has  improved  my  ability  to 
make  decisions  regarding  patient  care.  (Please  check  ONLY  one  response) 


Figure  8.  Improved  Decision-Making. 


I  I  I 


Effect  on  Reordering  of  ExamsilOa.  Immediate  PACS  image  access  to  films  performed  at 
Minersi'Meyersdale  has  reduced  the  number  of  exams  reordered  because  the  images 
previously  were  not  available  [lost  or  located  elsewhere)  when  I  needed  them.  [Please 
check  ONLY  one  response) 


25 


Figure  9.  Reduction  of  Duplicate  Exams. 

Questions  6-10  were  analyzed  using  the  original  scale,  a  5  level  Likert  scale,  and  an  aggregation 
(collapsing)  of  that  scale  comprised  of  3  levels: 


Answer  Choice  on  survey: 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

Not 

Applicable 

Answer  code  for  survey 
question: 

1 

2 

3 

4 

0 

Answer  code  for 
Aggregation: 

1 

2 

0 

Table  14.  Likert  Scale  Aggregation. 


Those  questions  related  to  Productivity  (6a,  6b,  7a,  9a,  10a)  individually  and  collectively 
indicated  an  overall  positive  (Agree)  response  as  demonstrated  by  both  the  bar  (column)  graphs 
and  means.  Likewise,  the  questions  (6c,  8a)  related  to  decision  making  also  indicated  an  overall 
positive  (Agree)  response,  both  individually  and  collectively. 
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Question 

O 

min 

MAX 

Mean 

6.  a)  RADIOLOGY  PACS  has  reduced  the  time  1  must  wait  to  review  an  image. 

55 

1 

4 

3.45 

6.  a)  aggregated:  Disagree,  Agree,  N/A 

55 

1 

2 

1.87 

6.  b)  1  access  images  more  frequently  with  RADIOLOGY  PACS  than  1  did  with  film. 

54 

0 

4 

3.35 

6.  b)  aggregated:  Disagree,  Agree,  N/A 

54 

0 

2 

1.76 

6.  c)  RADIOLOGY  PACS  has  facilitated  consultation  between  myself,  other 
clinicians  and/or  radiologists  at  other  healthcare  locations. 

55 

0 

4 

3.05 

6.  c)  aggregated:  Disagree,  Agree,  N/A 

55 

0 

2 

1.69 

7a.  My  productivity  has  improved  because  of  image  access. 

(Please  check  ONLY  one  response) 

55 

0 

4 

3.22 

7.  a)  aggregated:  Disagree,  Agree,  N/A 

55 

0 

2 

1.85 

Decision  Making: 

8a.  Immediate  PACS  image  access  has  improved  my  ability  to  make  decisions 
regarding  patient  care.  (Please  check  ONLY  one  response) 

55 

1 

4 

3.11 

8.  a)  aggregated:  Disagree,  Agree,  N/A 

55 

1 

2 

1.82 

Effect  on  Patient  Transfers: 

9a.  Immediate  PACS  image  access  has  reduced  the  number  of  patient  transfers 
between  facilities  due  to  the  ability  to  share  images  and  consult  remotely. 
(Please  check  one  response) 

55 

0 

4 

1.73 

9.  a)  aggregated:  Disagree,  Agree,  N/A 

55 

0 

2 

1.00 

Effect  on  Reordering  of  Exams: 

10a.  Immediate  PACS  image  access  to  films  performed  at  Miners/Meyersdale  has 
reduced  the  number  of  exams  reordered  because  the  images  previously  were 
not  available  (lost  or  located  elsewhere)  when  I  needed  them. 

54 

0 

4 

2.70 

10.  a)  aggregated:  Disagree,  Agree,  N/A 

54 

0 

2 

1.56 

Table  15.  Descriptive  Statistics. 
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Hypothesis  testing  of  those  questions  indicate  that  the  response  per  category  most  probably 
represented  a  real  difference  in  respondent  opinion.  With  the  exception  of  question  9,  the  results 
of  the  hypothesis  testing  did  not  change  for  the  aggregations  of  the  responses  of  those  questions. 
Question  9  yields  the  opposite  result  upon  aggregation  due  to  the  relatively  high  number  of  “Not 
Applicable”  responses.  [Asymptotic  significance  at  the  0.05  level  is  shown  for  all  hypothesis 
testing.] 


Null  Hypothesis 

Test 

Sig. 

Decision 

The  categories  of  6.  a) 

RADIOLOGY  PACS  has  reduced 
the  time  1  must  wait  to  review  an 
image,  occur  with  equal 
probabilities. 

One-Sample 

Chi-Square 

Test 

.000 

Reject  the 
null 

hypothesis. 

The  categories  of  6.  b)  1  access 
images  more  frequently  with 
RADIOLOGY  PACS  than  1  did  with 
film,  occur  with  equal  probabilities. 

One-Sample 

Chi-Square 

Test 

.000 

Reject  the 
null 

hypothesis. 

The  categories  of  6.  c) 

RADIOLOGY  PACS  has  facilitated 
consultation  between  myself,  other 
clinicians  and/or  radiologists  at 
other  healthcare  locations,  occur 
with  equal  probabilities. 

One-Sample 

Chi-Square 

Test 

.000 

Reject  the 
null 

hypothesis. 

The  categories  of  Productivity: 

7a.  My  productivity  has  improved 
because  of  image  access. 

(Please  check  ONLY  one  response) 
occur  with  equal  probabilities. 

One-Sample 

Chi-Square 

Test 

.000 

Reject  the 
null 

hypothesis. 

The  categories  of  Decision  Making: 
8a.  Immediate  PACS  image  access 
has  improved  my  ability  to  make 
decisions  regarding  patient  care. 
(Please  check  ONLY  one  response) 
occur  with  equal  probabilities. 

One-Sample 

Chi-Square 

Test 

.030 

Reject  the 
null 

hypothesis. 

The  categories  of  Effect  on  Patient 
Transfers: 

9a.  Immedicate  PACS  image 
access  has  reduced  the  number  of 
patient  transfers  between  facilities 
due  to  the  ability  to  share  images 
and  consult  remotely. 

(Please  check  one  response) 
occur  with  equal  probabilities. 

One-Sample 

Chi-Square 

Test 

.010 

Reject  the 
null 

hypothesis. 

The  categories  of  Effect  on 
Reordering  of  Exams: 

10a.  Immediate  PACS  image 
access  to  films  performed  at 
Miners/Meyersdale  has  reduced  the 
number  of  exams  reordered 
because  the  images  previously 
were  not  available  (lost  or  located 
elsewhere)  when  I  needed  them. 
(Please  che  occur  with  equal 
probabilities. 

One-Sample 

Chi-Square 

Test 

.000 

Reject  the 
null 

hypothesis. 

The  categories  of  9.  a)  aggregated:  One-Sample  Retain  the 

Disagree,  Agree,  N/A  occur  with  Chi-Square  .748  null 
equal  probabilities.  Test  hypothesis. 


Table  16:  Hypothesis  Testing 
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Additionally,  physicians  overwhelmingly  indieated  that  viewing  images  using  PACS  rather  than 
on  disk  (report  ineluded)  is  more  beneficial. 


18 


The  categories  of  12.  Is  viewing 
images  online  (via  Care  Portal  or 
standalone  PACS  viewers)  more 
beneficial  than  viewing  images  on  a 
disk  with  report  attached?  occur 
with  equal  probabilities. 


One-Sample 

Chi-Square 

Test 


.000 


Reject  the 
null 

hypothesis. 


Asymptotic  significances  are  displayed.  The  significance  level  is  .05. 


12.  Is  viewing  imaaes  online  (via  Care  Portal  or  standalone  PACS  viewers) 
more  beneficial  than  viewing  images  on  a  disk  with  report  attached? 


Figure  10.  Viewing  Images  -  Electronic  vs.  CD. 

Study  Conclusion: 

In  summary,  the  analysis  strongly  indicates  that,  overall,  CMMC  physicians  believed  that  the 
implementation  and  availability  of  PACS  at  the  rural  facilties  is  beneficial  relative  to 
productivity,  decision  making,  and  duplicate  testing.  Although  the  results  were  not  statistically 
significant,  a  7%  reduction  in  duplicated  chest  x-rays  and  CT  scans  of  the  head  (0-7  days)  is 
noteworthy  and  does  mildly  align  with  positive  physician  opinion.  Finally,  the  implementation 
of  PACS  technology  clearly  has  reduced  costs  and  patient  exposure  to  radiation. 


Study  Closure: 

This  study  has  been  completed  and  closed  with  Conemaugh  IRB  on  July  14,  201 1 .  Final  report 
and  supporting  documents  were  delivered  to  USAMRMC  ORP  HRPO  and  found  to  be 
acceptable  on  August  29,  2011  (refer  to  Appendix  3). 

Manuscript: 

A  study  manuscript  was  completed  and  submitted  to  Radiology  on  April  12,  2012. 

Unfortunately,  CMMC  received  notification  on  May  10,  2012  that  it  was  not  accepted  for 
publication.  Reviewer  comments  have  been  discussed  by  the  study  team  and  minor  edits  were 
completed  before  submission  to  Telemedicine  and  eHealth  in  July  2012.  The  manuscript  was  not 
accepted  for  publication  in  Telemedicine  and  eHealth.  The  study  team  is  again  reviewing 
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reviewer  eomments  and  is  in  the  proeess  of  selecting  another  journal  for  submission  for 
publication.  Please  refer  to  Appendix  4  for  a  copy  of  the  manuscript. 

Subtask  1.4  Deploy  (via  portal  technology)  a  pilot  demonstration  of  the  electronic  exchange  of 
private  sector  ambulatory  medical  records  with  the  DoD  and  other  selected 
stakeholders  using  test  data. 

The  second  phase  of  MIDHT  further  refined  the  CHS  Portal  capabilities  created  during  Phase 
One  to  better  align  with  the  Military  Health  System  (MHS)  strategic  goals  and  the  Nationwide 
Health  Information  Network  (NwHlN).  MIDHT  continued  to  identify  lessons  leamed/best 
practices  that  benefited  the  MHS  environment,  stakeholders  in  Southwestern  PA  and  private 
sector  hospitals/health  systems  nationwide. 

Authorization  to  Proceed  was  executed  with  subcontractor  Northrop  Grumman  (NG)  on  June  10, 
2009.  A  complete  Technical  Services  Agreement  was  executed  on  July  24,  2009  after  SOW 
review  by  LCDR  Steve  Steffensen. 

Project  representatives  attended  the  NwHIN  CONNECT  conference  on  June  29-30,  2009  and 
held  a  meeting  with  TATRC  leadership  to  discuss  future  work.  Project  representatives  viewed 
the  NwHIN  CONNECT  Release  2.1  webinar  on  July  28,  2009.  The  webinar  reviewed  changes 
made  to  CONNECT  code  in  v2.1,  as  well  as  reviewing  the  CONNECT  architecture.  NG  team 
members  initiated  meetings  with  the  TATRC  Advanced  Concepts  Team  (ACT)  and  ONC 
leadership  to  discuss  the  Virtual  Lifetime  Electronic  Record  (VEER)  -  San  Diego  development 
work  and  the  feasibility  of  using  the  Patient  Ancillary  Web  Services  (PAWS)  to  extract  data  from 
AHLTA.  Northrop  Grumman  representatives,  Steven  Clark  and  Allen  Barger,  attended  the 
CONNECT  “Code-A-Thon”  event  on  August  27,  2009.  The  event,  which  was  attended  by 
various  corporations  and  federal  organizations,  presented  interested  HIT  professionals  with  an 
opportunity  to  collaborate  with  the  CONNECT  development  team.  While  at  the  event,  NG 
representatives  participated  in: 

o  Downloading,  installing,  and  compiling  the  CONNECT  source  code 
o  Discussions  around  the  SoapUI  web  service  testing  software  and 

participation  in  demonstrations  on  how  to  write  tests,  create  messages, 
and  perform  specific  actions  in  both  the  free  and  professional  versions 
of  the  product 

o  Discussions  around  the  need  to  have  a  test  NHIN  platform  and  the 
benefits  that  would  be  realized  by  having  such  a  platform  to  test 
CONNECT  configurations  against 

o  Discussions  around  “best  practices”  for  CONNECT  development 

NG  configured  development  and  test  environments  in  Johnstown,  Salt  Lake  City  and  Chantilly, 
including  standing  up  a  DoD  and  CHS  gateway  with  CONNECT  v.  2.1.5  code  (open-source). 

NG  designed  VEER  spreadsheet  for  mappings  of  data  types  and  constructed  mappings  of  the 
PAWS  Connector  for  each  of  the  four  data  types: 
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o  Allergies:  PAWS  to  CAL,  CAL  to  C83 

o  Problems:  PAWS  to  CAL,  CAL  to  C83 

o  Patient  Info:  PAWS  to  CAL,  CAL  to  C83 
o  Medications:  PAWS  to  CAL,  CAL  to  C83 

On  November  16,  2009,  NG  delivered  a  fully  functional  code  drop  to  the  Federal  Healthcare 
Architecture  (FHA)  team,  thus  meeting  SOW  Deliverable  16.  This  drop  was  made  Yia  Dropbox, 
a  repository  used  by  the  TATRC/NG/FHA  developers.  In  addition  to  delivery  of  binary  code, 
Northrop  Grumman  also  provided  Phase  la  source  code  to  FHA,  with  the  intent  of  the  FHA  team 
folding  it  into  the  CONNECT  v2.L5  baseline  for  release  with  CONNECT  v2.3.  Northrop 
Grumman  provided  a  “Software  Design  Document”  and  “System  Installation  Guide”  to  the 
Defense  Health  Information  Management  System  (DHIMS)  and  FHA  representatives,  among 
other  documents.  NG  provided  considerable  talent  and  resources  to  the  Department  of  Defense, 
which  allowed  them  to  successfully  meet  VEER  timelines  and  objectives. 

The  ability  to  connect  to  the  CHS  Initiate  and  Allscripts  servers  (within  Conemaugh’s  data 
center)  was  verified  by  Northrop  Grumman.  Conemaugh  continued  to  integrate  and  deliver 
urgent  fixes  for  any  bugs  found  during  the  FHA  install  and  testing  process  for  VLER  la. 

Northrop  Grumman  adapted  their  clinical  viewer  for  MIDHT  use  and  integrated  application  with 
the  Universal  Inbox  for  correct  displaying  of  C32’s. 

John  Hargreaves,  Thomas  Simunich  and  Allen  Barger  represented  MIDHT  at  the  HIMSS 
conference  and  exhibition  during  the  first  week  of  March  2010  in  Atlanta,  GA.  This  provided  a 
good  opportunity  for  collaboration  with  military  and  government  representatives.  Our  successful 
demonstration  showed  a  bi-directional  health  information  exchange  of  test  C32  documents  (refer 
to  Appendix  5  for  sample  screenshots)  between  Conemaugh  and  DoD  utilizing  the  NwHIN 
CONNECT  v2.1.5  architecture. 

Fields  to  display  prefix,  suffix,  middle  initial  and  gender  were  added  to  the  Clinical  Viewer  GUI. 
Development  to  enable  pulling  this  information  back  from  the  adapter  was  completed. 
Development  was  completed  to  allow  the  Universal  Inbox  to  accept  an  encrypted  patient  ID. 
Work  was  completed  that  allowed  Universal  Inbox  to  be  “publicly”  viewed  (outside  of  the 
internal  NG  10-net).  Work  was  completed  to  remove  any  PHI  from  displaying  within  the 
Clinical  Viewer  URL.  NG  delivered  all  VLER  la  source  code  to  a  DHIMS  representative  on 
March  12  and  March  17,  2010. 

Allen  Barger  attended  the  CONNECT  Code-a-Thon  conference  in  Miami,  FE  from  April  27-29, 
2010,  where  he: 

o  Worked  with  the  TATRC  Advanced  Concepts  Team  (ACT)  on  a  fix  for  the  document 
return  latency  issue 

o  Spoke  to  participants  about  the  private  sector  implementation  of  the  document  assembler 
at  a  class  taught  by  members  of  the  TATRC  ACT  team 
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CONNECT  v2.4.1  was  subsequently  installed  on  the  local  Johnstown  development  environment 
and  the  local  environment  in  Salt  Lake  City,  Utah. 


Subtask  1.5  Perform  a  technical  feasibility  study  to  focus  on  repurposing  the  BHIE-AHLTA 
web  services  toward  the  existing  NHIN  Federal  Adapter  for  the  purpose  of 
standards  based  exchange  of  Military  Health  System  data  domains  with  private 
sector  partners. 

Deliverable  was  completed  on  September  30,  2009.  The  document  was  emailed  to  LCDR  Steve 
Steffensen  on  October  2,  2009. 


Subtask  1.6  Begin  development  on  a  private  sector  version  of  the  Federal  Gateway/ Adapter 

(work  to  be  based  on  the  code  that  is  anticipated  to  be  available  from  ONC)  using 
interoperable  HITSP  standards  to  progress  the  goals  of  this  national  effort. 

Despite  persistent  prodding,  the  Conemaugh  MIS  team  ultimately  concluded  that  Allscripts  was 
not  willing  to  assist  in  any  way  on  the  development  of  webservices  for  a  query-based  response  of 
a  Continuity  of  Care  Document  (CCD)  as  initially  planned.  An  XML-based  CCD  was  currently 
available  only  as  a  manual  “push.”  After  consultation  with  Northrop  Grumman,  the  MIS  team 
decided  to  utilize  the  CONNECT  “Document  Assembler”  approach  utilizing  HL7  messaging  to 
build  a  CCD  from  individual  data  domains  from  Allscripts  (patient  demographics,  allergies, 
medications  and  problems). 

Conemaugh  implemented  a  partial  Common  Access  Layer  Service  interface  to  return  to  a  remote 
partner  HL7  v3  clinical  document  architecture  (CD A)  documents  from  an  underlying  clinical 
information  system.  The  Common  Access  Layer  Service  Interface  web  service  was  developed  in 
Microsoft.NET.  AllScripts  TouchWorks  was  the  underlying  clinical  information  system  from 
which  data  was  provided.  No  application  programming  interface  existed  within  the  clinical 
information  system  by  which  to  return  information.  Therefore,  a  web  service  was  implemented  to 
retrieve  data  directly  from  the  clinical  system's  backend  database  via  a  mix  of  pre-existent  stored 
procedures  and  custom  database  queries.  Data  domains  included  active  patient  demographics, 
problems,  allergies  and  medications.  Additionally,  13  Conemaugh  test  patients  were  created  to 
match  the  selected  TATRC  CDR  patients.  Data  was  entered  into  the  Allscripts  EHR  for  testing 
purposes.  TATRC  arranged  AHLTA  VM  (test)  access  for  our  HIMSS  demo  with  FHA  for 
displaying  of  Conemaugh  data  within  AHLTA. 

In  addition,  the  Allscripts  Adapter  was  created: 

o  NG  development  team  worked  in  conjunction  with  CHS  personnel  to  stand-up  CHS 
Allscripts  Adapter 

o  NG  development  team  provided  CHS  personnel  with  appropriate  WSDL’s 
o  NG  development  team  created  connection  to  existing  CHS  Allscripts  Adapter 
o  Achieved  a  successful  return  of  care  records  from  the  CHS  Allscripts  system 
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Conemaugh  project  members  reviewed  and  added  specific  Adscripts  (version  11.1.7.283) 
documentation  to  the  “CHS  Adapter  Installation  and  Configuration  Guide”  created  by  Northrop 
Grumman.  In  addition,  all  related  source/binary  code  was  compiled  and  burned  onto  cd’s  for 
distribution  to  TATRC  on  April  7,  2010  (in  addition  to  NG  code). 

Conemaugh  representatives  held  an  introductory  phone  call  on  April  13,  2010  with  the 
Department  of  Veterans  Affairs  (Tim  Cromwell)  to  discuss  the  possibility  of  a  production  health 
information  exchange  pilot  in  SW  Pennsylvania.  Conemaugh  finalized  its  review/completion  of 
the  NHIN  Application  for  New  Participants  (including  DURSA).  Conemaugh  was  informed  of 
the  NHIN  On-boarding  process  and  required  conformance  testing. 


Subtask  1. 7  Perform  an  assessment  of  the  volume  of  cases  that  Conemaugh  physicians  have 
with  SSA  regarding  veteran/military  disability  claims  and  assess  provider 
satisfaction  with  existing  SSA  process  for  information  gathering  and  submission. 

The  provider  survey,  assessing  the  submission  process  of  medical  information  for  Social  Security 
disability  cases,  was  distributed  to  various  office  managers  within  the  Conemaugh  Health  System 
in  June  2010.  Fifty  one  (n=51)  providers  responded  to  the  survey  as  depicted  below. 


Please  describe  your  affiliation. 


I  MerTHjrial  Medical  Center 

I  Conemaugh 
'  Physician  Group 

I  Heyersdale 
'  Medical  Center 

I  Miners  Medical  Center 

I  Independent 
'  Physician  Practice 


Figure  1 1 .  Survey  Respondents  by  Affiliation. 
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Investigators  selected  the  following  questions  as  the  most  important  to  understand  the  current 
landscape  and  assess  future  provider  acceptance  of  an  electronic  record  submission  process  (e.g. 
Nationwide  Health  Information  Network)  to  the  SSA. 

Chi-Square  test  was  used  to  test  for  independence  between  question  response  choices. 
Independence  was  found  for  the  response  choices  for  Q3,  Q5,  Q6  and  Q9.  Independence  cannot 
be  stated  for  the  answer  choices  of  QIO.  The  Mann- Whitney  test  was  employed  to  investigate 
the  difference  between  the  responses  grouped  into  CHS  and  Independent  Physician  Practice.  No 
between  group  significance  was  found  for  any  question.  Additionally,  the  response  set  was 
grouped  into  CHS,  CPG,  and  Independent  Physician  Practice.  Using  the  Kruskal  Wallis  test,  no 
between  group  significance  was  found  for  any  question. 

In  order  to  increase  the  count  for  statistical  hypothesis  testing,  the  5 -choice  scale  contained  in  the 
original  survey  was  reduced  to  a  3 -point  scale  —  unsatisfied  (sum  of  very  unsatisfied  and 
unsatisfied),  neutral,  and  satisfied  (sum  of  satisfied  and  very  satisfied)  for  Q9.  Since  only  5 
responses  were  unsatisfied,  independence  (by  Chi-square  test)  was  found  between  that  choice 
and  each  of  the  other  two.  Next,  the  neutral  response  count  versus  that  of  satisfied  was  tested. 
No  independence  was  shown.  By  the  Mann-Whitney  test,  no  significance  was  found  when  the 
response  set  was  grouped  into  CHS  and  Independent  Physician  Practice.  Likewise,  the  Kruskal 
Wallis  test  did  not  show  significance  when  the  response  set  was  grouped  into  CHS,  CPG,  and 
Independent  Physician  Practice.  Therefore,  the  hypothesis  that  Conemaugh  providers  would 
prefer  an  electronic  system  for  submission  of  medical  records  to  the  SSA  rather  than  the  existing 
paper  system  was  not  accepted. 
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Question 

Answer  Options 

Response 

Percent 

Response 
Count,  (n) 

Very  unsatisfied 

2.0% 

1 

Unsatisfied 

8.2% 

4 

Neutral 

49.0% 

24 

Satisfied 

36.7% 

18 

9.  Please  rate  your  level  of  satisfaction  with  the 

Very  Satisfied 

4.1% 

2 

SSA  medical  record  submission  process. 

Reduction  from  5  choice  to  3  choice  Likert 
scaie 

Unsatisfied 

10.2% 

5 

Neutral 

49.0% 

24 

Satisfied 

40.8% 

20 

10.  Would  you  personally  use  an  electronic 

Yes 

34.8% 

16 

processing  system  for  submitting  medical 
records  that  would  result  in  significantly 

No 

23.9% 

11 

reduced  SSA  determination  times  for  your 
patients  (e.g.  Nationwide  Health  Information 
Network)? 

Need  More 
Information 

41.3% 

19 

Table  17.  SSA  Survey  Responses. 
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Arm  2:  The  Impact  of  Consumer  Informatics  in  the  Chronic  Care 
Model:  Metabolic  Syndrome  and  Gestational  Diabetes  in  a 
Rural  Setting  (A-15835.1) 


Subtask  2.1  Deploy  HIE  tools  for  patient  and  community  outreach  in  varied  rural 
environments. 


Subtask  2.2  Research  and  evaluate  the  impact  of  a  personal  health  record  (PHR)  on 

provider(s)  and  consumer(s)  with  particular  focus  on  chronic  disease  prevention. 

Subtask  2.3  Research  and  evaluate  the  impact  of  web-based  secure  messaging,  online 

consultations,  prescription  renewals,  and  appointment  scheduling  on  consumer 
awareness  and  their  ability  to  effectively  self  manage  their  health  compared  to 
those  consumers  not  using  a  PHR. 

Background 

Chronic  Disease 


The  burden  of  chronic  disease  in  the  United  States  is  great  and  will  increase  over  time. 

Prevention  of  these  conditions  is  essential  in  order  to  increase  the  health  and  quality  of  life  of  the 
aging  population  while  decreasing  the  burden  on  the  healthcare  system.  In  Pennsylvania  (PA), 
chronic  disease  is  the  leading  cause  of  death  and  disability,  accounting  for  80  percent  of  health 
care  costs  and  hospitalizations.  Due  to  the  increasing  cost  of  health  care,  many  families, 
governments,  businesses,  and  insurers  have  decided  to  limit  health  care  services,  and  as  a  result 
only  56  percent  of  people  with  a  chronic  condition  received  evidence-based  care  that  is 
recommended  for  their  conditions. 

Chronic  disease  management,  if  adopted  by  health  care  providers  and  patients,  will  have  long¬ 
term  effects  on  health  benefits  for  residents.  As  a  result,  there  will  also  be  economic  benefits 
since  healthier  employees  will  mean  improved  productivity  for  the  businesses  as  well  as  a 
reduced  need  for  emergency  room  visits  and  lower  insurance  premiums.  Diabetes  is  a  chronic 
condition  which  affects  the  national  and  global  populations  as  well.  As  one  of  the  main  causes  of 
death  in  many  developed  countries,  diabetes  places  a  substantial  burden  on  the  world.  The  rates 
of  type  1  and  type  2  diabetes  are  climbing,  making  this  disease  one  of  the  major  challenges  for 
healthcare  professionals.  The  American  Diabetes  Association  reports  that  there  are  25.8  million 
children  and  adults  in  the  United  States,  or  8.3%  of  the  population,  who  have  diabetes,  and 
another  79  million  with  “prediabetes”."^  A  2010  report  states  that  8.8%  of  Pennsylvanians  have 
been  told  by  a  doctor  they  have  diabetes,  ranking  the  state  eighteenth  in  the  nation.^ 

Many  researchers  have  found  positive  outcomes  with  the  use  of  computers  in  disease  education.*’" 

^  Gustafson  et  al.  found  that  computer-based  patient  education  resulted  in  improved  cognitive 
functioning  and  social  support,  in  addition  to  an  increase  in  life  activities  and  more  active 
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participation  in  healthcare.  Further,  with  computer-based  education,  patients  were  more  likely  to 

O 

call  a  doctor  for  problems,  but  physician  office  visits  were  less  frequent  and  shorter  in  duration. 

Rural  patients  have  higher  rates  of  developing  chronie  illness  and  exhibit  poorer  health 
behaviors.  There  are  many  barriers  to  delivering  eare  to  rural  eommunities,  including:  resource 
limitations,  serving  in  a  low-volume  environment,  challenges  with  reeruiting  health  care 
professionals,  and  difficulty  paying  for  and  implementing  neeessary  technology.^  Additional 
barriers  to  rural  health  care  include:  long  distances  to  travel  (30  miles  +),  inclement  weather 
eonditions,  time,  family  and  work  eommitments,  economic  barriers,  and  a  large  elderly 
population  who  are  unwilling  or  unable  to  drive  to  eentral  loeations  are  key  factors  that  prevent 
them  from  accessing  existing  resourees. 

Metabolic  Syndrome 

One  of  the  main  precursors  of  diabetes  is  a  diagnosis  of  metabolic  S5mdrome  (MS).  MS  is 
defined  as  a  grouping  of  interrelated  risk  factors  that  include  elevated  blood  pressure  and 
elevated  plasma  glucose  levels.  One  theory  states  that  insulin  resistance  is  the  main  cause  of  MS, 
thus  the  high  correlation  of  MS  to  type  2  diabetes  mellitus.^®  A  previous  study,  based  on 
information  from  the  2003-2006  National  Health  and  Nutrition  Examination  Survey,  estimated 
the  prevalence  of  MS  to  be  34%  of  U.S.  adults  or  104  million  people  based  on  2010  census 
data.^^  People  with  MS  are  also  believed  to  be  at  an  increased  risk  for  cardiovascular  disease. 

The  key  emphasis  in  the  MS  population  is  on  the  management  of  modifiable  risk  factors 
(including  obesity,  physical  inactivity,  smoking,  and  atherogenic  diets)  through  lifestyle 
modifications.'^  However,  with  current  trends  in  shortening  the  length  of  physician  visits,  there  is 
not  sufficient  time  to  provide  education  on  diet,  exercise,  and  healthy  behaviors.  In  this  situation, 
education  is  most  often  the  aspect  of  the  office  visit  that  gets  disregarded.  Patients  are  left 
without  the  knowledge  they  need  to  manage  their  condition  at  home.  There  is  a  need  to  provide 
reliable  diabetes  education  to  patients  who  experience  the  repercussions  of  time  limitations  in 
doctors’  offices.  Therefore,  a  goal  in  diabetes  education  is  to  provide  patients  with 
comprehensive  disease  education,  without  increasing  the  time  of  a  physician’s  office  visit.'"* 

Gestational  Diabetes 

Shaw  et  al.  studied  the  interest  and  satisfaction  of  pregnant  women  who  had  aeeess  to  online 
health  information  and  records  in  Hamilton,  Ontario.  The  two  arm  study  provided  pregnaney 
related  information  through  a  website  and  electronie  access  to  antenatal  reeords  for  enrolled 
subjeets.  Utilization  data  showed  that  women  with  aeeess  to  personal  information  were  much 
more  active  than  the  other  group  who  only  had  access  to  general  information.  Pregnant  women 
are  believed  to  have  more  motivation  to  engage  in  positive  health  behaviors  and  providers  spend 
a  lot  of  time  answering  questions  verbally  with  this  population.  With  the  growing  trend  of 
shorter  clinical  encounters,  patients  will  most  likely  have  questions  they  did  not  have  time  to  ask 
during  their  visit.  The  researchers  eoneluded  that  the  pregnant  population  is  prepared  to  use 
online  health  information  to  improve  their  well-being.'^ 
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Gestational  diabetes  (GD)  is  eommonly  defined  as  glucose  intolerance  first  recognized  during 
pregnancy  and  prevalence  may  be  as  high  as  20%  of  pregnancies.  Perinatal  complications 
include  hypertension,  preterm  birth  and  caesarian  deliveries  among  others.  Postpartum 
complications  for  the  mother  may  include  diabetes  and  cardiovascular  disease.  Monitoring  of 
glucose  levels,  weight  management,  nutritional  intake,  physical  activity  and  pharmacotherapy 
can  reduce  health  problems  during  and  after  pregnancy. Although  an  acute  condition, 
gestational  diabetics  may  benefit  from  additional  education  and  greater  access  to  their  providers. 
Gestational  diabetics  were  added  as  an  additional  study  arm  in  201 1. 

Personal  Health  Records  (PHRs) 

President  Barack  Obama  has  provided  a  large  infusion  of  funding  towards  transforming  the 
health  care  delivery  system  through  new  investment  in  health  information  technology,  such  as 
electronic  medical  records  (EMRs),  electronic  PHRs  and  health  information  exchange.  Many 
stakeholders  have  become  engaged  with  PHRs  in  the  last  few  years,  including  consumers, 
hospitals/health  systems,  physicians,  government  (e.g.  Department  of  Defense,  Department  of 
Health  &  Human  Services  (HHS),  Medicare),  insurance  companies  (e.g.  Aetna),  web  companies 
(e.g.  Google),  employers  (e.g.  Wal-Mart),  and  information  technology  vendors.  It  remains  to  be 
seen  if  momentum  can  be  maintained  to  have  uniform  standards  to  create  a  more  efficient  and 
less  costly  health  care  industry  through  wide  PHR  usage. 

Electronic  PHR  adoption  by  consumers  and  providers  has  been  slow  in  the  United  States  due  to 
varying  factors.  According  to  a  2008  consumer  survey  of  1,850  adults  commissioned  by  the 
Markle  Foundation,  only  2.7%  of  U.S  adults  (6.1  million  persons)  utilize  electronic  PHRs.  A 
more  recent  2010  California  Healthcare  Foundation  report  states  that  only  7%  of  U.S.  adults  use 
some  sort  of  PHR,  with  51%  of  access  offered  by  their  health  insurance  plan'^.  The  most  cited 
reasons  for  a  lack  of  interest  were  concerns  about  the  protection  of  medical  information  and  how 
it  would  be  shared.  To  combat  such  concerns.  Connecting  for  Health,  a  public -private 
collaborative  group  engaging  more  than  100  organizations  representing  all  major  stakeholders  of 
the  healthcare  industry,  published  a  Common  Framework  for  Networked  Personal  Health 
Information.  This  framework  provides  a  roadmap  for  establishing  trust  among  all  participants 
and  encourages  the  appropriate  treatment  of  personal  health  information  when  electronically 
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shared  across  various  distinct  networks. 

Many  PHRs  have  similar  functions  and  services  that  enable  consumers  to  manage  their  health  by 
compiling  their  personal  health  information  from  varied  sources  (e.g.  medications,  lab  results, 
allergies,  surgeries,  office  visits,  conditions,  family  histories,  etc)  in  one  place.  Many  popular 
PHRs  are  electronic,  which  makes  them  available  anywhere  and  anytime  there  is  an  available 
connection  to  the  Internet.  This  ease  of  access  is  especially  important  in  an  emergency  situation, 
and  could  ultimately  help  save  a  life.  Since  the  consumer  controls  the  PHR,  the  following  data 
may  be  added  as  well:  food  allergies,  health  conditions,  over  the  counter/herbal  medications, 
physician  list,  and  emergency  contacts. 

Through  secure  messaging  with  their  physician,  patients  can  virtually  ask  questions  about  their 
health,  saving  them  time  away  from  work  and  traveling  costs  to  the  office  as  done  traditionally. 
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Secure  messaging  can  be  extremely  helpful  in  situations  that  involve  non-urgent  issues,  which 
will  likely  free  up  time  for  office-based  appointments  with  more  serious  issues.  Primary-care 
office  visits  dropped  25%  from  2004-2007  after  Kaiser  Permanente’s  Hawaii  region  offered 
secure  messaging  between  providers  and  patients.  A  more  recent  patient-centered  care 

initiative  gaining  momentum  is  called  “Blue  Button”.  It  allows  Veterans  to  download  their 
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personal  health  information  from  the  Veterans  Administration  and  share  with  their  doctors. 

The  concept  has  also  been  implemented  by  CMS  for  Medicare  beneficiaries.  It  is  likely  the 
technology  will  penetrate  the  private  sector  and  reach  more  patients  in  the  coming  years. 

A  Deloitte  2009  Survey  of  Health  Care  Consumers  stated  that  57%  of  respondents  want  a  secure 
Internet  site  that  would  enable  them  to  access  their  medical  records,  schedule  office  visits,  refill 
prescriptions,  and  pay  medical  bills.  Additionally,  55%  of  respondents  are  interested  in  email 
access  with  their  doctor  and  42%  of  respondents  are  interested  in  a  PHR  that  is  connected  to  their 
doctor’s  office.  It  is  evident  that  electronic  PHRs  can  help  close  the  gap  between  consumers’ 
expectations  and  what  they  are  actually  receiving  from  physicians. 

PHRs  may  also  help  patients  who  struggle  with  chronic  diseases  through  self-management  tools. 
For  example,  some  PHRs  have  preventive  service  reminders  and  educational  materials  that  allow 
patients  to  better  control  their  conditions  through  online  tools  (i.e.  diabetes  management). 

Methods 

The  specific  aims  of  this  protocol  were  to  develop  an  innovative  approach  in  providing  chronic 
disease  health  information  to  both  a  metabolic  S5mdrome  and  gestational  diabetes  populations 
residing  in  a  primarily  rural  geographical  area  and  to  evaluate  subsequent  effects  on  health 
outcomes.  The  primary  objective  was  to  assess  any  differences  in  health  outcome  measures  (e.g. 
weight,  glucose)  between  those  who  utilize  the  PHR  (Relay  Health  was  the  specific  product,  see 
Figure  1)  and  those  who  continue  using  normal  communication  methods  as  part  of  the  regular 
medical  management  of  their  condition  (Control  Group).  Secondary  objectives  were  to 
determine  usage  and  satisfaction  of  PHR  functionalities  (e.g.  secure  messaging)  from  the  patient 
and  provider  perspectives.  Subjects  in  the  PHR  Group  were  provided  with  a  customized 
instructional  manual. 
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O  RelayHealth  -  Welcome  to  RelayHealth  -  Microsoft  Internet  Explorer  provided  by  Conemaugh  Health  System 
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Figure  12.  Relay  Health  Personal  Health  Record. 
Hypotheses 


MS  Arm 

There  will  be  a  significant  difference  in  the 
following  health  outcome  measures  between 
the  PHR  group  and  the  Control  Group: 

•  Blood  Pressure 

•  Body  Fat 

•  Body  Mass  Index 

•  Glucose 

•  HAlc 

•  HDL 

•  Triglycerides 

•  Waist  Circumference 

•  Weight 


GD  Arm 

There  will  be  a  significant  difference  in  the 
following  health  outcome  measures  between  the 
PHR  group  and  the  Control  Group: 

•  Blood  Pressure 

•  HAlc 

•  Weight 
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Study  Design 


The  study  team  sought  to  enroll  100  subjeets  (MS  arm)  and  74  subjects  (GD  arm)  in  the  study. 
Subjects  were  a  sample  of  English-speaking  patients  who  met  inclusion  criteria  for  the  study 
(defined  below).  Subjects  meeting  inclusion  criteria  were  randomized  into  one  of  two  groups:  a 
PHR  Group  or  a  Control  Group.  In  addition,  participating  providers  utilizing  the  PHR  were 
asked  to  provide  input  through  a  survey  instrument.  The  study  enrolled  subjects  from  2010- 
2012. 


Inclusion  and  Exclusion  Criteria 

MS  Arm  —  Inclusion: 

•  Subject  must  meet  three  of  the  following  five  criteria  as  assessed  by  research  staff 
to  be  included  in  the  study: 

•  Blood  Pressure  equal  to  or  greater  than  130/85  or  the  person  is  receiving 

pharmacologic  therapy  for  hypertension 

•  Waist  circumference: 

■  Men  greater  than  40  inches 

■  Women  greater  than  35  inches 

•  Triglycerides  equal  to  or  greater  than  150  mg/dL  or  the  person  is  receiving 

pharmacologic  therapy  for  elevated  triglyceride  levels 

•  Fasting  Blood  glucose  equal  to  or  greater  than  100  mg/dL  or  the  person  is 

receiving  pharmacologic  therapy  for  elevated  glucose  levels 

•  HDL  Cholesterol: 

■  Men  less  than  40  mg/dL 

■  Women  less  than  50  mg/dL 

•  Subject  must  sign  the  informed  consent  document  and  HIPAA  authorization 

•  Subject  must  commit  to  attend  all  three  data  collection  points 

•  Subject  must  have  Intemet  connectivity 

•  Subject  must  be  willing  to  use  the  PHR  and  agree  to  its  “Terms  of  Use” 

•  Subjects  must  be  an  established  patient  of  a  study  physician 

•  Subject  must  be  English-speaking  and  literate 

MS  Arm  -  Exclusion: 

•  Persons  who  are  under  18  years  of  age 

•  Women  who  are  pregnant,  since  metabolic  syndrome  in  pregnancy  is  managed 
very  differently  than  for  other  adults 

•  Persons  who  are  not  willing  to  sign  the  informed  consent  document  and  HIPAA 
authorization 

•  Persons  who  have  no  access  to  the  Intemet 

•  Persons  who  are  not  English-speaking  and  literate 


Page  37  of  99 


GD  Arm  -  Inclusion: 

•  Subject  has  gestational  diabetes  diagnosis 
confirmed  by  Conemaugh  OB/GYN 

•  Subject  must  sign  the  informed  consent 
document  and  HIPAA  authorization 

•  Subject  must  commit  to  attend  standard 
visits  with  physician  and/or  midwife 

•  Subject  must  have  Internet  connectivity 

•  Subject  must  be  willing  to  use  the  PHR  and 
agree  to  its  “Terms  of  Use” 

•  Subject  must  be  an  established  patient  of  a 
study  physician  and/or  midwife 

•  Subject  must  be  English-speaking  and 
literate 

•  Subject  must  be  16  years  of  age  or  older 


GD  Arm-  Exclusion: 

•  Persons  who  are  under  16  years  of  age 

•  Persons  who  are  not  willing  to  sign  the 
informed  consent  document  and  HIPAA 
authorization 

•  Persons  who  have  no  access  to  the  Internet 

•  Persons  who  are  not  English-speaking  and 
literate 


Data  collection 

The  research  team  used  existing  outcome  measures  from  the  previous  MIDHT  study  to  evaluate 
health  outcomes  of  a  metabolic  S5mdrome  population,  including  weight,  BMI,  waist 
circumference,  body  fat,  blood  pressure.  Hemoglobin  Ale,  glucose,  HDL  and  triglycerides.  This 
was  done  to  identify  clinical  outcomes  of  patients  who  were  involved  in  the  study.  The  team  also 
used  a  technology  evaluation  to  assess  the  comfort  level  and  satisfaction  of  using  a  PHR,  both 
from  the  patient  and  provider  perspective. 

Subjects  enrolled  in  the  gestational  diabetes  arm  had  their  data  retrospectively  provided  (by  chart 
review)  to  the  research  team  by  Conemaugh  OB/GYN  staff.  The  data  included  weight  and  blood 
pressure  at  three  time  points  (28-32  weeks,  last  visit  before  delivery  and  6  weeks  postpartum).  In 
addition,  oral  glucose  tolerance  tests  were  conducted  at  two  time  points  (28-32  weeks  and  6 
weeks  postpartum).  This  did  not  apply  to  Hemoglobin  Ale,  which  was  ordered  by  the  Principal 
Investigator  at  the  time  of  screening,  within  12  hours  of  delivery,  and  6  weeks  postpartum. 
Subjects  in  this  arm  were  provided  the  same  survey  to  assess  satisfaction  of  the  PHR. 

Additionally,  a  chart  review  was  conducted  in  which  the  following  data  was  collected: 

❖  Demographic  information:  age,  race,  and  history  of  gestational  diabetes  were  to  have  been 
analyzed  for  correlation  with  the  outcome  variables  -  sample  size  did  not  permit  this  analysis. 

❖  Weight  and  blood  pressure  at  each  standard  of  care  clinical  appointment  post  diagnosis  of 
gestational  diabetes  were  collected  to  calculate  trends  of  these  variables;  the  expanded  dataset 
offering  the  potential  for  a  higher  degree  of  confidence  of  calculated  statistical  measures. 

❖  Gestational  age,  baby  weight  and  length,  number  of  babies  birthed,  neonatal  outcomes: 
through  a  nonparametric  correlation  analysis,  these  data  were  to  have  been  used  as  additional 
indicators  of  the  degree  of  control  of  subject  gestational  diabetes. 
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Data  collection  occurred  according  to  the  following  table: 


Participant 

Approach^ 

Group^ 

Instrument 

Time  Point 

1 

(Baseline) 

2 

3 

Monthly 

Sub j  ects 

Both 

All 

Readiness  to 

Change 

X 

Sub j  ects 

Both 

All 

Concomitant 
Medication  Log 

X 

X 

X 

Sub j  ects 

Quant . 

All 

Data  collection 
from  Subject 

Charts  and  Lab 

work 

X 

X 

X 

Physicians 

Both 

All 

PHR  Survey 

X 

Sub j  ects 

Both 

I 

PHR  Survey 

X 

Sub j  ects 

Both 

All 

Outside  Study 
Exposure 

X 

X 

X 

Both 

Quant . 

I 

Usage  Report 

X 

Sub j  ects 

Both 

All 

Internet  Usage 
Assessment 

X 

Note  1:  Approach  =  qualitative  (Qual.),  quantitative  (Quant.),  or  Both 

Note  2:  Group  =  I  for  Intervention,  All  for  both  Intervention  and  Control  Groups 


Table  18.  Data  Collection  Schedule. 
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Readiness  to  Change  Instrument 

A  readiness  to  change  tool  was  completed  by  each  subject  prior  to  beginning  the  study.  The  tool 
is  based  on  the  Transtheoretical  Model  (TTM),  proposed  by  Prochaska  and  DiClemente  in  1983. 
The  model  of  behavior  change  has  been  used  to  create  effective  interventions  that  promote  health 
behavior  changes.  TTM  focuses  on  each  individual’s  decision  making  and  involves  emotions, 
cognition,  and  behavior.  The  TTM  describes  basic  Stages  of  Change  in  which  individuals 
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identify  themselves  based  on  their  current  status  and  readiness  to  change. 

The  results  of  the  readiness  to  change  scale  were  compared  to  the  final  outcomes  of  the  study,  to 
determine  if  there  was  a  correlation  between  each  subject’s  readiness  to  change  their  diet  and 
exercise  and  the  actual  outcomes  that  are  made  as  a  result  of  the  study. 

PHR  Patient  Survey 

Subjects  in  the  PHR  Group  were  asked  for  their  individual  input  regarding  their  satisfaction  and 
usage  of  the  personal  health  record  used  during  the  study.  The  survey  was  mainly  based  upon 
questions  provided  by  Relay  Health,  which  have  been  used  previously  with  patients  on  a  national 
level.  Relay  Health  provided  their  permission  to  use  the  survey  questions.  Subjects  completed 
the  survey  at  the  end  of  their  participation.  The  survey  was  created  in  “Survey  Monkey”  and  was 
made  available  to  subjects  online. 

PHR  Provider  Survey 

Study  physicians  were  asked  for  their  written  input  regarding  their  satisfaction  and  usage  of  the 
personal  health  record  used  during  the  study.  The  survey  was  mainly  based  upon  questions 
provided  by  Relay  Health,  which  have  been  used  previously  with  physicians  on  a  national  level. 
Relay  Health  provided  their  permission  to  use  the  survey  questions.  The  survey  was  created  in 
“Survey  Monkey”  and  was  made  available  to  providers  via  interoffice  mail. 

Outside  Study  Exposure 

All  subjects  were  asked  to  describe  their  exposure  to  education  (e.g.  websites)  and  resources  (e.g. 
office-based  physician  visits)  regarding  their  metabolic  syndrome  or  gestational  diabetes 
condition  “OUTSIDE”  of  the  PHR  during  the  study  period  using  an  internally  developed 
instrument.  Additionally,  subjects  provided  information  on  their  exercise  and  nutrition  habits. 
The  information  collected  was  used  for  control  purposes  to  help  analyze  the  true  impact  of  the 
PHR. 

Medication  Log 

Subjects  were  asked  to  provide  a  list  of  their  medications  (e.g.  name,  dosage,  start  date  and/or 
end  date)  during  all  study  visits.  The  self-reported  information  was  documented  by  study 
coordinators.  This  information  is  part  of  the  subjects'  medical  history  and  may  have  potential 
effects  on  outcome  measures  (e.g.  blood  pressure,  lipids  levels). 
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PHR  Usage  Report 

PHR  usage  for  each  specific  subject  was  manually  collected  by  the  Associate  Investigator  via  the 
Relay  Health  application  on  a  monthly  basis.  He  had  administrator  access  and  viewed  the 
provider/subject  communications  and  blood  glucose  input  data  to  perform  simple  message  counts 
and  health  record  completion.  This  information  was  used  to  support  the  secondary  objective  of 
determining  which  functionality  was  used  and  how  often  by  study  stakeholders. 

Internet  Usage  Assessment 

An  internally  developed  questionnaire  was  deployed  to  assess  subjects’  usage  on  the  Internet 
before  the  study  commenced.  Results  helped  the  study  team  determine  internet  usage,  what 
online  activities  were  being  performed,  their  experience  with  online  health/medical  information, 
and  previous  usage  of  an  electronic  personal  health  record. 

Recruitment/Enrollment 

In  order  to  generate  patient  interest  in  the  study,  Conemaugh  staff  used  various  recruiting 
strategies  during  the  study  period,  including: 

■  Direct  patient  letters 

■  Advertisements  in  numerous  newspapers 

■  Facebook  postings 

■  Flyers  and  posterboards  in  participating  physician  offices 

■  Phone  On  Hold  messages 

■  Global  emails 

■  Face-to-face  meetings  with  physicians 

■  Community  health  fairs 

■  Metabolic  syndrome  lectures 

■  Direct-to-consumer  study  introduction 

Subject  recruitment  for  the  respective  study  was  challenging.  Interest  for  the  MS  arm  was 
greatest  from  patients  deriving  from  primary  care  suburban  physician  offices  however  anecdotal 
concerns  persist  regarding  Intemet  access  and  a  high  elderly  population  in  rural  communities. 
Additional  sites  were  added  to  the  study  with  the  goal  of  increasing  enrollment.  These  included 
the  Johnstown  Free  Clinic  (patients  without  health  insurance)  and  the  Conemaugh  Weight 
Management  Center  (patients  involved  in  a  structured  program).  Overall,  interest  from  these 
populations  was  minimal  and  compliance  was  an  issue  as  well.  A  total  of  66  subjects  were 
enrolled  in  the  MS  Arm  with  45  subjects  completing  the  study. 

Over  40  patients  with  GD  were  approached  by  physicians  and  research  staff  about  participating 
in  the  study.  Unfortunately  a  large  majority  of  patients  were  not  interested  in  participating  as 
only  five  (5)  subjects  were  enrolled  in  the  GD  Arm  with  3  subjects  completing  the  study.  This 
population  was  likely  consumed  with  their  new  diagnosis  and  current  condition  leading  to  their 
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general  feeling  of  not  wanting  to  add  additional  responsibilities  to  their  life.  A  current  literature 
search  did  not  yield  any  research  studies  involving  the  pregnant  population  and  PHRs. 

The  following  table  describes  study  enrollment,  including  inquiries,  screenings,  number  of 
enrolled,  number  of  withdrawals  and  number  of  subjects  completing  the  study. 


Study  Enrollment  Summary 


Description 

Study  Arm:> 

GD 

MS 

Inquiries 

40 

160 

Screenings 

5 

78 

Screen  Failures 

0 

12 

Withdrawals 

2 

19 

Enrolled: 

5 

66 

Completed  (see  note): 

3* 

45* 

(combined  sums  for  both  Study  Arms)  Totals 

Study  Group  (by  randomized  group  assignment):> 

Control: 

PHR: 

Completed  (at  least  chronologically):> 

19 

29 

*  NOTE: 

A  completed  subject  is  defined  as  one  that  both  chronologically  completed 
the  study  and  on  which  complete  data  exists  per  the  protocol  design. 


The  data  completeness  varies  both  by  subject  and  variable  (e.g.  survey 
question  response,  physical  measurement,  lab  value,  etc.);  this  fact 
naturally  leads  to  the  potential  for  differing  but  appropriate  and  valid  sample 
sizes  for  each  conducted  analysis. 


Table  19.  Study  Enrollment  Summary. 


Page  42  of  99 


Results  and  Data  Analysis 

Disposition  of  Study  Hypotheses: 

MS  Arm 


This  hypothesis  is  rejected  for  all  outcome  measures  except  blood  pressure. 

Inherent  in  the  study  hypotheses  was  that  the  access  and  subsequent  use  of  a  PHR  would  improve 
the  outcome  measures  for  the  PHR  group  to  a  greater  degree  than  any  improvement  observed  in 
the  Control  group.  In  spite  of  both  groups  remaining  sufficiently  consistent  over  time  on  the 
assessed  potential  confounders  and  statistical  significance  for  blood  pressure,  two  realities  of  the 
study,  (1)  the  confounding  of  the  PHR  group  due  to  the  influence  of  medication  changes  during 
the  study;  (2)  an  extremely  low  PHR  usage,  require  complete  rejection  of  the  study  hypothesis. 
Although  47  subjects  chronologically  completed,  data  from  one  subject  (Control  group)  was 
deficient  to  a  degree  that  rendered  it  unusable  for  analysis  on  any  of  the  outcome  measures.  One 
subject  (Control  group)  had  data  for  lab  values  but  not  physical  measurements.  Therefore,  the 
final  maximum  sample  size  available  for  analysis  consisted  of  either  45  or  46  subjects, 
depending  upon  the  outcome  measure. 

GD  Arm 


This  hypothesis  could  not  be  tested  due  to  a  very  small  sample  size  (due  to  data  incompleteness, 
N=3). 


Demographics  by  Study  Arm  and  Study  Group: 


Gender  *  Group  *  Study  Arm  Crosstabulation  Age  category  *  Group  *  Study  Arm  Crosstabulation 

Count  Count 


Study  Arm 

Group 

Total 

Control 

PHR 

Gestational  gender  Female 
Diabetes  jotal 

2 

2 

3 

3 

5 

5 

X  1  I-  ^  Female 

Metabolic  gender  , 

Syndrome 
^  Total 

19 

12 

31 

22 

13 

35 

41 

25 

66 

Study  Arm 

Group 

Total 

Control 

PHR 

^  16 -29  years 

Aqe 

Gestational  category  30  -  39  years 
Diabetes  40  -  49  years 

Total 

1 

2 

3 

1 

1 

2 

1 

3 

1 

5 

16-29  years 

30  -  39  years 

X  1  I-  Age  40 -49  years 

SyndrorSe  50  -  59  years 

60  -  69  years 

70  yrs  and  older 

Total 

2 

3 

6 

10 

8 

2 

31 

1 

4 

7 

11 

9 

3 

35 

3 

7 

13 

21 

17 

5 

66 

Table  20A.  Gender  Demographics. 


Table  20B.  Age  Demographics. 
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Results  of  Within  and  Between-Group  Analysis  of  Outcome  Measures: 

The  outcome  measures  (physical  measurements  and  lab  values)  were  analyzed  using  a  repeated 
measures  analysis  of  variance  (RM-ANOVA)  over  the  three  time  points  (baseline  [1],  three 
months  [2],  six  months  [3])  on  a  within  and  between-group  basis.  The  family-wise  alpha  was  set 
at  0.05  and  the  Sidak  correction  was  used  to  adjust  for  multiple  comparisons.  The  groups  being 
assessed  were  Control  and  PHR.  Computation  of  inferential  statistics  was  only  possible  for  the 
MS  Arm  of  the  study.  The  GD  Arm  contained  only  five  subjects  -  only  three  with  nearly 
complete  data. 

Calculations  for  the  MS  arm  produced  the  following: 

Within-Group  Results  (N  =  46,  maximum): 

For  the  Controls,  the  within-group  results  showed  no  statistical  significance  on  any  of  the 
outcome  measures.  The  results  for  PHR  showed  statistical  significance  on  waist  circumference 
(inches),  weight  (lbs.),  and  BMI  only.  Pairwise  comparisons  revealed  the  statistically  significant 
change  to  be  between  baseline  and  six  months.  For  all  three  variables,  the  direction  of  change 
was  negative  (last  minus  first)  and  indicates  an  improvement  of  the  outcome  measure. 


Variable 

N,  Control 
// 

N,  phr 

Multivariate 

p-value 

Mauchly’s 

Test 

Epsilon  Corrected 
p-value, 

Greenhouse-Geisser 

Pairwise 

p-value 

Mean 

Diff, 

(6  mos.  - 
baseline) 

waist  circumference 

18// 27 

0.012 

Passed 

N/A 

0.011 

-1.390 

weight 

18// 27 

0.041 

Violated 

0.010 

0.032 

-4.977 

BMI 

18// 27 

0.053 

Violated 

0.014 

0.043 

-0.737 

Table  21.  Within-Group  p- values. 

Between-group  Results  (N  =  46,  maximum): 

The  between-group  (between  Control  and  PHR)  results  showed  no  statistical  significance  on  any 
of  the  outcome  measures  except  for  both  the  systolic  and  diastolic  blood  pressure  values.  On 
average,  the  PHR  group  diastolic  and  systolic  pressures  were  less  than  the  Control  group  by 
-1 1 .6  mm  HG  and  -5.4,  respectively.  Box’s  Test  held  for  both  variables  and  the  p-values  were 
0.006  and  0.005,  respectively.  For  these  outcome  measures  the  sample  sizes  were  18  (Control) 
and  27  (PHR).  The  following  graphs  depict  the  change  of  these  variables  over  time  by  group. 
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Estimated  Marginal  Means  of  BP,  systolic 


Estimated  Marginal  Means  of  BP,  diastolic 


Group 

(randomized) 

- Control 

- PHR 


W  85- 


(0 

0) 


70- 


"I - 1 - T 

1  2  3 

time 


Figure  13.  Estimated  Marginal  Means  of  Blood  Pressure,  systolic  and  diastolic. 


Calculations  for  the  GD  arm  follow: 


Although  inferential  statistics  are  non-calculable  for  the  GD  arm  due  to  insufficient  sample  size, 
the  change  or  delta  between  time  points  (delta  equals  as  described  in  the  ‘where’  section  found 
on  the  following  tables)  for  the  five  subjects  over  the  outcome  measures  does  provide  some 
useful  information.  Of  course,  extrapolation  to  any  group  or  population-wide  conclusions  is  not 
feasible.  The  raw  data  has  been  provided  for  additional  clarity. 


The  delta  for  the  outcome  measures  for  the  five  available  subjects  are  as  follows: 


Raw  Data 

BP_s_ 

BP_s_ 

BP_d_ 

BP_d_ 

weight_ 

weight_ 

1 

7 

1 

7 

1 

7 

96 

124 

64 

80 

187 

174 

104 

102 

76 

60 

174 

150 

112 

96 

60 

60 

143 

130 

102 

100 

60 

68 

204 

190 

110 

100 

78 

64 

204 

181 

104.8 

104.4 

67.6 

66.4 

182.2 

164.8 

6.419 

11.171 

8.764 

8.295 

25.171 

24.811 

96 

96 

60 

60 

143 

130 

112 

124 

78 

80 

204 

190 

Deltas 

variable 

BP_s_ 

BP_d_ 

wt_7_1 

HA1c_ 

HA1c_ 

HA1c_ 

nanne:» 

7_1 

7_1 

D_S 

P_D 

P_S 

28 

16 

-13.0 

-0.2 

A/C 

A/C 

-2 

-16 

-24.5 

0.1 

0 

0.1 

-16 

0 

-13.5 

0 

0.4 

0.4 

-2 

8 

-13.5 

-0.1 

0.2 

0.1 

-10 

-14 

-22.5 

0.3 

A/C 

A/C 

mean 

-0.4 

-1.2 

-17.4 

0.0 

0.2 

0.2 

std.  dev. 

16.935 

13.828 

5.617 

0.192 

0.200 

0.173 

min 

-16 

-16 

-24.5 

-0.2 

0 

0.1 

max 

28 

16 

-13 

0.3 

0.4 

0.4 

Table  22.  Gestational  Diabetes  Arm  Outcomes  Measures,  deltas  and  raw  data. 
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Raw  Data 

variable 

name:» 

HA1c_0_ 

Screening 

HA1c_1_ 

Delivery 

HA1c_2 

_Post 

Baby 

Weight 

lbs 

Baby 

Length_ 

inches 

OGT_ 

2hr 

OGT_ 

3hr 

4.9 

4.7 

M 

8.375 

21.0 

M 

120 

5.1 

5.2 

5.2 

6.625 

19.0 

86 

167 

5.2 

5.2 

5.6 

5.750 

19.0 

M 

M 

6.0 

5.9 

6.1 

6.438 

19.5 

187 

M 

5.3 

5.6 

M 

6.313 

19.0 

73 

M 

mean 

^So? 

19.5 

std.  dev. 

0.418 

0.455 

0.451 

0.992 

0.866 

62.405 

33.234 

min 

4.9 

4.7 

5.2 

5.750 

19.0 

73 

120 

max 

6 

5.9 

6.1 

8.375 

21.0 

187 

167 

Table  23.  Gestational  Diabetes  Arm  Outcomes  Measures,  raw  data  continued. 


where: 

NC  =  not  computable  M  =  missing 

BP_s;  systolic  blood  pressure,  mmHG  wt;  weight,  lbs. 

BP_d:  diastolic  blood  pressure,  mmHG  HAlc:  Hemoglobin  Ale 

OGT:  Oral  Glucose  Tolerance  Test 

7  refers  to  the  6  month  time  point  and  1  to  the  baseline  time  point 
S  refers  to  screening,  D  to  At  Delivery,  and  P  to  six  weeks  postpartum 


These  calculations  show  signs  of  a  generally  decreasing  trend  (improvement)  over  all  the 
outcome  measures  except  delta  HAlc  between  the  post  and  screening  time  points.  HAlc  shows 
the  opposite,  an  increasing  trend.  However,  the  increasing  trend  does  not  place  any  subject 
within  the  range  considered  to  be  clinically  diabetic. 

Reearding  Subject  Medications: 

Information  regarding  prescribed  medications  was  complete  for  the  same  45  subjects  that  had 
complete  data  for  all  outcome  measures.  For  this  analysis,  complete  refers  to  (the  existence  of) 
data  at  both  the  beginning  and  end  of  the  study.  Relative  to  the  beginning  and  end  of  the  study, 
these  subjects  were  categorized  as  having  a  net  medication  change  or  not.  Medication 
information  was  self-reported  by  subjects  without  any  insight  regarding  medication  adherence. 
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General  Medication  Class: 

Hypertensive 

Lipid  Lowering 

Glucose  Control 

Study  Time  point: 

Baseline 

6  months 

Baseline 

6  months 

Baseline 

6  months 

Subjects  Identified  as  having  NO  net  medication  change  from  the  start  to  the  end  of  the  study 

N,  Control  =  15;  N,  phr  =  19 

Count  of  subjects  with  at  least  one  med. 

23 

23 

23 

23 

8 

7 

Count  of  subjects 

34 

34 

34 

34 

34 

34 

%  of  subjects  with  at  least  one  med. 

68% 

68% 

68% 

68% 

24% 

21% 

Average  of  Baseline  &  6  mo. 

68% 

68% 

22% 

Subjects  Identified  as  having  a  net  medication  change  from  the  start  to  the  end  of  the  study 

N,  Control  “  5;  N,  phr  =  8 

Count  of  subjects  with  at  least  one  med. 

10 

7 

6 

5 

8 

7 

Count  of  subjects 

11 

11 

11 

11 

11 

11 

%  of  subjects  with  at  least  one  med. 

91% 

64% 

55% 

45% 

73% 

64% 

Average  of  Baseline  &  6  mo. 

77% 

50% 

68% 

NOTE:  Total  N,  control  =18;  Total  N,  phr  =  27;  Total  N  for  this  analysis  =  45 


Table  24.  Categorical  Medication  Analysis. 


The  details  of  the  1 1  subjects  identified  as  having  a  net  medication  change  are: 

The  Control  Group  details: 

♦  one  subject  eliminated  only  lipid  lowering  medication, 

♦  one  subject  eliminated  one  glucose  control  medication,  and 

♦  one  subject  eliminated  two  (all)  glucose  control  medications. 

The  PHR  Group  details: 

♦  four  subjects  eliminated  one  hypertensive  medication 

o  three  reduced  from  one  to  none 
o  one  reduced  from  two  to  one 

♦  one  subject  added  a  hypertensive  medication 

♦  one  subject  added  a  lipid  lowering  medication 

♦  one  subject  eliminated  one  of  three  glucose  control  medications 

♦  one  subject  added  two  glucose  control  medications 

Obviously  medication  changes  could  have  confounded  the  study  results  since,  if  effective,  the 
medications  alone  could  cause  or  at  least  contribute  to  any  observed  change  in  the  outcome 
measures,  particularly  the  lab  values.  Given  this  truth,  the  question  then  follows  as  to  whether 
the  percentage  of  those  subjects  with  a  net  change  was  sufficient  to  impact  the  between  group 
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analysis.  Given  30%  of  the  PHR  group  had  a  net  medieation  change,  a  confounding  influence 
can  reasonably  be  presumed  to  have  existed. 

PHR  Usage  Summary; 

Twenty  nine  subjects  assigned  to  the  PHR  group  actually  utilized  the  application.  Irrespective  of 
study  completion,  forty  subjects  were  assigned  to  the  PHR  group.  Ten  did  not  finish 
participation  in  the  study.  Of  the  thirty  subjects  in  the  PHR  group  that  finished,  28  used  the 
PHR.  One  non-finisher  from  the  GD  arm  did  have  usage  data  and  an  equivalent  time  of 
opportunity  for  usage  (three  months)  as  the  one  other  GD  arm  subject  that  did  finish  the  study 
and  was  included  in  this  analysis.  All  subjects  had  a  six  month  time  frame  in  which  to  use  the 
PHR.  The  following  table  represents  the  sum  of  unique  activity  (total  usage)  for  all  subjects  by 
message  type  and  health  record  section. 


N,  total: 

29  subjects 

Health  Record  Data  Types 

Secure  Messaging  Data  Types 

PHR_ 

Mecls_ 

AIgs 

PHR_ 

Fam_ 

Soc_ 

Hx 

PHR_ 

Probs 

JProc 

PHR_ 

Results 

PHR_ 

Imuz 

PHR_ 

Vitals 

PHR_ 

Files 

PT_MSG 

_Rx_Refl 

_Req 

PT_MSG 

_N2D 

PT_MSG 

_TL_Res 

_Req 

PT_MSG 

_Admin 

PT_MSG_ 

Apt_Req 

PT_MSG 

_PTJnit_ 

webV 

PT_MSG 

_Ref 

SUM 

22 

13 

10 

7 

7 

1 

1 

18 

12 

11 

6 

3 

1 

0 

percentage 

36% 

21%0 

16% 

11%0 

11%0 

2% 

2% 

35% 

24% 

22% 

12% 

6% 

2% 

0% 

cumulative 

percent 

36% 

57% 

74%o 

85% 

97% 

98% 

100% 

35% 

59% 

80% 

92% 

98% 

100% 

100% 

Health  Record  Data  Types 

- 

Secure  Messaging  Data  Types 

Variable  Name 

Description 

Variable  Name 

Description 

PHR_Mecls_Algs 

Mods  and  Allergies 

PT_MSG_Rx_Refl_Req 

Rx  Refill  Request 

PHR_Fam_  Soc_Hx 

Family  &  Social  History 

PT_MSG_N2D 

Note  to  Doc 

PHR_Probs_Proc 

Problems  and  Procedures 

PT_MSG_TL_Res_Req 

Test/Lab  Results  Request 

PHR_Results 

Results 

PT_MSG_Admin 

(general)  Administration 

PHRJmuz 

Immunizations 

PT_M  S  G_Apt_Req 

Appointment  Requests 

PHR_Vitals 

Vitals 

PT_MSG_PT_lnit_webV 

Patient-Initiated  webVisits 

PHR_Files 

Files 

PT_MSG_Ref 

Referrals 

Table  25.  Personal  Health  Record  Usage. 
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Patient  PHR  Survey  Summary; 


The  following  tables  highlight  that  the  subjeets  had  a  positive  PHR  experienee:  88%  were 
satisfied  or  very  satisfied;  88%  perceived  PHR  to  be  secure  &  confidential;  46%  said  they  would 
continue  to  use  a  PHR  after  the  end  of  the  study.  Also,  66%  accessed  the  PHR  from  home  and 
76%  used  a  DSL  internet  connection.  Recall,  not  all  subjects  eligible  to  respond  to  this  survey 
instrument  did  so;  and  not  all  of  those  who  responded,  answered  every  question.  Therefore  the 
total  response  (sample  size)  per  question  varies. 


Age  category  *  gender  Crosstabulation 


Age  category*  Overall  Satisfaction  of  Relay  Health  Crosstabulation 


Count  Count 


age  category 

Overall  Satisfaction  with  the  Relay 
Health  service 

Total 

Very  Satisfied 

Satisfie 

d 

Dissatisfied 

18-29  years 

0 

1 

0 

1 

30  -  39  years 

1 

3 

0 

4 

40  -  49  years 

5 

0 

0 

5 

50  -  59  years 

2 

4 

2 

8 

60  -  69  years 

3 

1 

1 

5 

70  yrs  or  older 

1 

1 

0 

2 

Total 

12 

10 

3 

25 

age  category 

gender 

Total 

Male 

Female 

18 -29  years 

0 

1 

1 

30  -  39  years 

1 

3 

4 

40  -  49  years 

1 

4 

5 

50  -  59  years 

1 

7 

8 

60  -  69  years 

2 

3 

5 

70  yrs  or  older 

1 

1 

2 

Total 

6 

19 

25 

The  answer  ehoiee,  very  dissatisfied,  was  not  ehosen  by  any  respondent. 

Table  26.  Age*gender  crosstab.  Table  27.  Age*overall  satisfaction  crosstab. 


Crosstab 

Did  you  feel  your  medical  information  and 
personal  communications  were  secure 


and  confidential  over  the  Internet? 

Count 


age  category 

Total 

Yes 

Not 

Sure 

18 -29  years 

1 

0 

1 

30  -  39  years 

4 

0 

4 

40  -  49  years 

5 

0 

5 

50  -  59  years 

7 

1 

8 

60  -  69  years 

3 

2 

5 

70  yrs  or  older 

2 

0 

2 

Total 

22 

3 

25 

The  answer  ehoiees,  ‘no’  and  ‘I  had  problems’,  were  not 
ehosen  by  any  respondent. 

Table  28.  PHR  Security  &  Confidentiality. 


Crosstab 

A  Personal  Health  Record  (PHR)  allows  you  to 
organize  and  store  all  of  your  health  and  medical 
information  in  one  convenient  location.  Will  you 
continue  using  a  PHR  after  the  study  is  over? 


Count 


age  category 

Total 

Yes 

No 

Not 

Sure 

18-29  years 

0 

0 

1 

1 

30  -  39  years 

2 

0 

2 

4 

40  -  49  years 

3 

0 

1 

4 

50  -  59  years 

5 

1 

2 

8 

60  -  69  years 

1 

1 

3 

5 

70  yrs  or  older 

0 

0 

2 

2 

Total 

11 

2 

11 

24 

Table  29.  PHR  Usage  Post  Study. 


Page  49  of  99 


Crosstab 


Count 


In  what  ways  has  Relay  Health  helped  you?  (check  all  that  apply) 

age  category 

Gave  me 
greater 
access  to 

Improved 

communication 

Easier  to 
obtain 
medication 
refills 

Saved 

me 

Easier 

to 

obtain 

Easier  to 
obtain  my 
medical 
information 

My  questions 
and  concerns 

were 

Improved 

my 

medical 

care 

my 

doctor/staff 

with  my  doctor 

time 

lab/test 

results 

addressed 
more  quickly 

18-29  years 

1 

1 

30  -  39  years 

1 

3 

2 

1 

1 

1 

40  -  49  years 

2 

2 

2 

2 

1 

1 

1 

50  -  59  years 

4 

3 

4 

1 

B 

2 

60  -  69  years 

4 

5 

1 

D 

1 

2 

70  yrs  or  older 

1 

1 

B 

2 

Total 

12 

10 

10 

8 

8 

7 

3 

2 

Table  30.  PHR  Benefits  -  Patient  Perspective. 
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Provider  PHR  Survey  Summary; 


Five  providers  responded  to  the  survey  but  did  not  complete  all  questions. 

Recall,  not  all  providers  eligible  to  respond  to  this  survey  instrument  did  so;  and  not  all  of  those 
who  responded,  answered  every  question.  Therefore  the  total  response  (sample  size)  per 
question  varies. 


5- 


5- 


4- 


3- 


O 

O 


2- 


Defintely  would  Probably  would  Not  sure  Probably  would  Definitely  would 

not  not 

Likelihood  to  Recommend  Relay  Health  to  other  physicians 


Figure  14.  Provider  Perceptions  of  Relay  Health. 


Which  of  the  following  Relay  Health  features  were  most  valuable  to  your  practice? 


N 

webVisit 

Patient 

health 

record 

Send  education/ 
resources  to  specific 
paitent  populations 

Note  to 
doctor 

Send  lab/  test 
results  to  patients 

Appointment 

scheduling 

Valid 

0 

0 

0 

2 

4 

1 

Missing 

5 

5 

5 

3 

1 

4 

Sum 

2 

4 

1 

How  has  Relay  Health  benefited  your  medical  practice? 


Enhanced 

N 

Enhanced  QoC 
through  improved 
communication 
with  patient 

Improved 

patient 

satisfaction 

Improved 

office 

efficiency 

Increased 
time  for 
patient  care 

patient  care 
due  to 
access  to 

accurate 

Improved  office 
staff  -  physician 
communication 

Decreased 
telephone  call 
volume 

Reduced 
unnecessary 
office  visitis 

info. 

Valid 

2 

0 

0 

0 

0 

0 

0 

0 

Missing 

3 

5 

5 

5 

5 

5 

5 

5 

SUM 

2 

Table  3 1 .  Clinical  Benefits  of  PHR  -  Provider  Perspectives. 


Page  51  of  99 


Readiness  to  Change: 

An  independent  t-test  was  attempted  for  a  between  arm  (GD  vs  MS)  analysis  by  considering  the 
answer  choices  to  both  questions  of  the  Readiness  to  Change  Survey  as  interval  data  types.  The 
first  question  asks  about  the  subject’s  current  physical  activity  status  and  the  second  about 
current  diet  status.  The  answer  choices  for  both  questions  are  coded  from  1  to  5  with  the 
following  designations: 

1  =  pre-contemplation  3  =  preparation  5  =  maintenance 

2  =  contemplation  4  =  ACTION 

The  MS,  N  =  66,  sample  size  is  approximately  13x  that  of  GD,  N  =  5.  This  sample  disparity 
violates  symmetry  assumptions  and  contributes  to  issues  arising  from  heterogeneity  of  variance. 
Levene’s  Test  is  violated  for  current  Diet  Status.  Both  questions  violate  normality.  Despite 
these  issues,  the  difference  between  the  group  means  of  current  diet  status  yields  a  statistically 
significant  p-value  of  0.005  at  alpha  =  0.05.  However,  this  finding  should  be  viewed  with 
extreme  caution  given  the  aforementioned  issues.  As  such,  a  Mann- Whitney  U  test  was 
performed  and  revealed  no  statistical  significance  for  either  question  and  thereby  substantiated 
that  the  violations  are  too  severe  for  application  of  analysis  by  parametric  algorithms. 


Internet  Usage  Survey; 

Collectively,  the  following  graphs  demonstrate  sufficient  similarity  between  Control  and  PHR 
groups  by  study  arm  regarding  computer  and  internet  browsing  experience,  internet  usage 
(time/day),  and  location  of  said  use,  such  that  homogeneity,  at  least  on  these  parameters,  is 
presumed  to  have  existed.  Also,  the  total  count  of  surveyed  internet  activities  over  the  six  month 
period  is  fairly  normal. 


Sample  Size  Summary 


Variable  Description 

Control  Group 

PHR  Group 

Study  Arm 

Computer  Experience 

Internet  Browsing  Experience 

Count  of  Unique  Internet  Activities 

Internet  Usage/day  -  Personal 

Internet  Usage/day  -  Work/Business 

3 

2 

GD 

30 

35 

MS 

NOTE: 


♦  The  total  N  for  either  study  arm  does  not  exceed  enrollment  for  that  arm 

♦  Data  was  missing  for  one  subject  that  did  not  finish  the  study 

♦  Usable  data  was  present  for  some  subjects  that  did  not  finish  the  study 


Table  32.  Sample  Size  for  Internet  Usage.  Survey. 
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Count  Count  Count  Count 


Group 


Control  PHR 

22- 
20- 
18- 
16- 
14- 
12- 
10- 
8- 
6- 
4- 
2- 


years  years  years  years  years  years 

IU_computer  experience  (time)  IU_computer  experience  (time) 


1  I  I 


Group 


Control  PHR 


22- 

20- 

18- 

16- 

14- 

12- 

10- 

8- 

6- 

4- 


lUJnternet  browsing  experience  lUJnternet  browsing  experience 


Figure  15.  Computer  and  Internet  Browsing  Experienee. 
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Gestational  Diabetes  Metabolic  Syndrome  Gestational  Diabetes  Metabolic  Syndrome 


Group 


Control 


PHR 


lUJnternet  activities,  total  count  lUJnternet  activities,  total  count 
of  unique  activities  of  unique  activities 


Figure  16.  Histogram  of  Unique  Internet  Aetivities. 


Group 


Control 


PHR 


22- 
20- 
18- 
16- 
•£  14- 

g  12- 

O  10- 
8- 
6- 
4- 


day  day  day 

lUJnternet  usage/day,  nominal  - 
Personal 


I 


day  day  day 

lUJnternet  usage/day,  nominal 
Personal 


Figure  17A.  Internet  Usage  -  Personal. 
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Gestational  Diabetes  Metabolic  Syndrome  Gestational  Diabetes  Metabolic  Syndrome 


Group 


Control 


PHR 


<  1  hr/  day  1  -  4  hrs/  4  -  8  hrs/  >  8  hrs/ 
day  day  day 

lUJnternet  usage/day,  nominal 
Work/Business 


"I —  — \ —  — \ — - 1 - 

<  1  hr/  day  1  -  4  hrs/  4  -  8  hrs/  >  8  hrs/ 
day  day  day 

lUJnternet  usage/day,  nominal 
Work/Business 


O 

(D 

(/) 

!-► 

fi) 

o' 

3 

fi) 


&) 

O' 

0 

0 

0 


Q) 

O' 

o 

o' 

O) 
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o 


CO 

c 

Q. 
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Figure  17B.  Internet  Usage  -  Work/Business. 


Outside  Exposure  Survey; 

The  following  questions,  or  sums  therein,  from  the  Outside  Exposure  Survey  were  analyzed 
using  a  RM-ANOVA  over  six  contiguous  months  (this  survey  was  to  be  completed  every  month) 
on  a  within  and  between-group  basis.  The  family-wise  alpha  was  set  at  0.05  and  the  Sidak 
correction  was  used  to  adjust  for  multiple  comparisons.  The  groups  being  assessed  are  Control 
(N  =  15)  and  PHR  (N  =  23).  Computation  of  inferential  statistics  was  only  possible  for  the 
Metabolic  Syndrome  Arm  (MS)  of  the  study.  The  Gestational  Diabetes  Arm  (GD)  contained 
only  five  subjects  -  three  with  complete  data. 

♦  QOl :  (sum  therein):  Total  count  of  unique  education  resources  by  month 

♦  Q02:  Categorized  number  of  websites  visited,  variable  treated  as  interval 

♦  QOS:  Categorized  current  exercise  habits,  variable  treated  as  interval 

♦  Q04:  (sum  therein):  The  sum  of  unique  dietary  changes  by  month 

No  statistically  significant  change  over  time  (within  group)  or  between  groups  was  found. 
Because  some  of  the  variables  were  presumed  interval  and  not  scale  as  is  preferred  for  RM- 
ANOVA,  the  within-group  analysis  was  also  accomplished  using  the  Friedman  Test.  Those 
results  did  not  change  the  conclusions  drawn  from  the  RM-ANOVA  analysis.  Note  that 
currently  a  non-parametric  equivalent  to  a  between-group  RM-ANOVA  does  not  exist. 
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Therefore,  stability  over  time  and  between  groups  was  presumed  to  have  existed  for  the  outside 
exposure  variables  and  as  sueh  produeed  no  eonfounding  influenee  on  the  study  analysis. 

Discussion 

When  compared  to  previous  published  studies  on  PHR  usage,  our  experience  with  recruitment 
challenges  during  this  prospective  pilot  research  study  was  likely  unique  due  to  our  small,  rural 
geographic  service  area.  The  number  of  prospective  studies  analyzing  PHR  usage  within  a 
similar  organizational  and  geographic  scope  to  ours  is  quite  limited.  For  organizations  of  similar 
size  and  scope,  testing  PHRs  prior  to  wide  deplo5mient  is  a  critical  success  factor;  our  experience 
indicates  that  both  process  and  organization  behavioral  changes  will  probably  be  warranted  in 
order  to  improve  interest  from  subjects  and  providers  alike.  Future  research  should  focus 
attention  on  the  current  state  of  health  information  technology  implementation  within  the 
organization  as  this  will  impact  provider  willingness  and  support.  Simply,  the  wider  the  gap 
between  the  current  state  and  one  with  a  fully  functioning  and  highly  utilized  HIT 
implementation  will  inform  to  the  barriers  and  guide  the  transformation.  Participation  in  our 
study  and  usage  of  the  Relay  Health  application  by  providers  was  directly  influenced  by  the 
implementation  of  electronic  health  records  (EHR).  As  the  nation  focuses  on  “meaningful  use” 
stage  1  requirements  for  EHRs,  consumer-facing  technology  is  not  a  priority  for  many  healthcare 
organizations  at  this  time.  In  reality,  the  implementation  and  support  of  PHRs  is  likely  five  years 
away  as  support  for  it  from  additional  “meaningful  use”  criteria  for  patient-centric  initiatives 
materializes.  Multiple  issues  still  need  to  be  resolved  regarding  PHRs,  including  provider 
reimbursement,  service  costs,  data  ownership  and  accuracy,  security  of  PHI  online,  response 
time  expectations,  etc. 

Successful  PHR  applications  and  portals  available  from  large  insurance  companies,  medical 
centers  and  the  Department  of  Veterans  Affairs  (e.g.  myHealtheVet)  have  built-in  incentives  for 
subjects  to  participate.  These  include  efficient  tethered  access  to  scheduling  and  clinical  data 
contained  within  the  electronic  health  record  (in  addition  to  secure  messaging),  wellness 
reminders  based  upon  age  and  gender  and  email  alerts  about  upcoming  appointments.  Health 
summaries  are  also  a  valuable  tool  that  you  can  share  with  a  new  doctor.  Implementers  should 
decide  on  their  primary  objectives  and  determine  if  they  will  purchase  an  off-the-shelf  product 
(e.g.  Relay  Health)  or  develop  internally.  Attitudes  and  perceptions  may  be  drastically  different 
in  the  private  sector  compared  to  a  military/veteran  setting.  PHRs  may  produce  benefits  as  a 
communication  tool  but  may  have  little  impact  on  management  of  conditions  and  related 
outcomes  measurements  as  the  results  of  our  study  suggests. 

Another  area  of  concern  is  technology  comprehension  by  subjects.  As  you  work  with  subjects 
who  have  varying  levels  of  education  and  literacy  and  technical  experience,  providers  must 
understand  that  “one  size  does  not  fit  all.”  Our  experience  showed  that  despite  providing  a 
detailed,  written  User  Manual  (at  or  below  8th  grade  level)  to  all  subjects  in  paper  and  electronic 
form,  some  subjects  did  not  comprehend  how  to  use  certain  functionality  (e.g.  webVisit)  and 
therefore  did  not  utilize  the  application  to  its  fullest  extent.  Subjects  were  also  hesitant  to  ask  for 
help  or  contact  the  vendor  if  they  had  questions  or  problems.  However,  subjects  overwhelmingly 
did  feel  their  health  information  was  secure  online  (88%).  This  finding  is  opposite  of  a  frequent 
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criticism  of  PHRs  in  general. 


The  three  year  study  elearly  highlighted  a  general  laek  of  interest  in  PHRs  by  subjeets,  regardless 
of  condition,  in  our  geographic  area  as  highlighted  by  persistent  recruitment  ehallenges  and 
system  usage.  Various  strategies  were  used  to  gamer  interest  with  little  sueeess,  which  aligned 
with  provider  disinterest.  For  those  subjeets  that  had  a  chance  to  utilize  the  technology,  very  few 
used  it  as  a  self-improvement  tool  to  enhance  aecess  to  their  healtheare  team  and  to  utilize  online 
educational  resources.  PHRs  potentially  may  benefit  medication  adherence  through  greater  self- 
monitoring  of  lab  results,  efficient  medication  refills  and  greater  access  to  providers  through 
secure  messaging.  Our  findings  generally  align  with  other  studies  that  experienced  limited  PHR 
use  and  had  a  difficult  time  associating  usage  with  improved  health  outcomes.^^'^^ 

Study  Closure 

Study  enrollment  was  elosed  on  September  24,  2012.  This  was  done  in  order  to  complete  data 
eolleetion,  data  analysis  and  write-up  of  results  before  the  contraet  end  date  of  January  3 1,  2013. 
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KEY  RESEARCH  ACCOMPLISHMENTS 


■  Northrop  Grumman  support  of  White  House  mandated  Virtual  Lifetime  Electronic 
Record  (VLER)  la  project  in  San  Diego,  CA 

■  Northrop  Grumman  contribution  of  document  assembler  code  to  the  FHA  CONNECT 
release  v2.4;  Conemaugh  contribution  of  Allscripts  adapter  code 

■  Successful  bi-directional  health  information  exchange  (using  test  data)  between 
Department  of  Defense  and  Conemaugh  utilizing  the  NwHESl  model 

■  Full  disclosure  of  technical  documentation  and  code  (source/binary)  to  TATRC 

■  The  following  table  summaries  NG  deliverables  that  have  been  completed 


No. 

Title 

Due  Date 

4 

Demonstration  of  Electronic  Exchange  of  Test  Allscripts  CCD 
via  CHS  Portal 

Completed  4/1/2010 

5 

Technical  and  Functional  Requirements  -  Phase  II 

Completed  2/23/2010 

6 

Technical  Feasibility  Study  -  Migrating  AHLTA/BHIE  Web 
Services  Towards  NHIN 

Completed  9/30/2009 

7 

NHIN  Background/Gap  Analysis 

Completed  2/22/2010 

8 

Migration  Plan  for  Movement  from  Viewable  (Textual)  Data  to 
Computable  Data 

Completed  1/15/2010 

9 

Electronic  Patient  Consent  Assessment  Report 

Completed  9/30/2009 

10 

Demonstration  of  the  NHIN  Adaptor  With  One  Site 

Completed  4/1/2010 

11 

Assist  with  DURSA  Submission 

Completed  1/21/2010 

12 

Disclosure  of  Technical  Specifications 

Completed  4/3/2010 

13 

Checklist  of  Criteria  Met/Unmet  Necessary  to  Meeting  HIE 
Accreditations  and  Certifications 

Completed  3/12/2010 

14 

MIDHT  Portal  Test  Plan 

Completed  3/19/2010 

15 

Inventory  List 

Completed  1/25/2010 

16 

VLER  Code  Drop  to  FHA/DHIMS 

Completed  1/18/2010 

Table  33:  Northrop  Grumman  Deliverables. 


■  Completion  and  closure  of  study  A  -  1 5835.2 

■  Completion  and  closure  of  study  A  -  1 6 1 92 . 1 

■  Completion  of  study  A  -15835.1,  closure  documents  sent  to  CMMC  IRB 
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REPORTABLE  OUTCOMES 


■  Conemaugh  presented  MIDHT  project  at  a  TATRC-sponsored  workshop  at  St.  Francis 
University  on  September  3,  2009. 

■  John  Hargreaves’  presentation  on  personal  health  records  at  the  Cambria-Somerset 
Council  Aging  Conference  -  Champion,  PA  on  October  29,  2009. 

■  Health  Information  Exchange  Demonstration  @  HIMSS  -  Atlanta,  GA  (March  1-3, 
2010). 

■  John  Hargreaves’  presentation  on  personal  health  records  at  the  Cambria-Somerset 
Council  Conference  at  the  Slopes  in  Champion,  PA  on  March  5,  2010. 

■  John  Hargreaves  and  Charlie  Shaw  presented  the  MIDHT  project  at  the  TATRC  Product 
Line  Review  in  Frederick,  MD  on  March  23,  2010. 

■  MIDHT  studies  highlighted  at  the  6*  Annual  Conemaugh  Research  Poster  Symposium  - 
Johnstown,  PA  on  March  22-26,  2010. 

■  Northrop  Grumman  contribution  of  the  “document  assembler’’  code  to  the  nationally 
available  FHA  CONNECT  release  2.4  (open-source). 

■  Virtual  Lifetime  Electronic  Record  la  project  tasks  completed  by  Northrop  Grumman. 

■  Allen  Barger’s  presentation  at  the  CONNECT  Code-A-Thon  in  Miami,  FL  on  April  28- 
29,  2010. 

■  Joe  Dado  and  John  Hargreaves  presented  at  the  TATRC  Product  Line  Review  on  March 
15,  201 1  in  Falls  Church,  VA. 

■  MIDHT  studies  highlighted  at  the  7th  Annual  Conemaugh  Research  Poster  S5miposium  - 
Johnstown,  PA  on  March  28  -  April  1,  201 1. 

■  Conemaugh  presented  research  update  to  Dr.  Steve  Steffensen  and  Betty  Levine  during 
local  site  visit  in  Johnstown,  PA  on  June  9,  201 1. 

■  MIDHT  studies  highlighted  at  the  8th  Annual  Conemaugh  Research  Poster  S5miposium  - 
Johnstown,  PA  on  March  26-30,  2012. 
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DoD  Adapter 

Conemaugh  Adapter 

Jar  Files 

AdapterDocumentAssemblyProxyE  JB  .j  ar 

AdapterDocument  AssemblyProxyE  JB  .j  ar 

BOSServiceEndpointProviderEJB.jar 

BOSServiceEndpointProviderE  JB  .j  ar 

AdapterCommonDataLayerEJB.jar 

DocumentManagerE  JB  .j  ar 

DocumentManagerE  JB  .j  ar 

DocumentRepositoryE  JB  .j  ar 

DocumentRepositoryE  JB  .j  ar 

DoDConnector  .j  ar 

CHSConnector.jar 

MpiEJB.jar 

MpiEJB.jar 

NHIN  Adapters  erviceE  JB  .j  ar 

NHIN  Adapters  erviceE  JB  .j  ar 

NhincHL?  J  axbLib  .j  ar 

NhincHL?  J  axbLib  .j  ar 

Properties  Files 

Adapter common datalayer.properties 

Adapter.properties 

Adapter.properties 

DoDconnector.properties 

CHSconnector.properties 

Repository.properties 

Repository.properties 

SQL  Scripts 

Docassemblydll.sql 

Docassemblydll.sql 

Docrepositorydll.sql 

Docrepositorydll.sql 

T  emplatedbdll.  sql 

T  emplatedbdll.  sql 

WSDL  Files 

BOSServiceEndpointProvider.wsdl 

BOSServiceEndpointProvider.wsdl 

AdapterCommonDataLayer.wsdl 

DocumentAssembly.wsdl 

DocumentAssembly.wsdl 

DocumentManager.wsdl 

DocumentManager.wsdl 

Doc  ViewerRequestS  ervices  S  ervice .  wsdl 

Doc  ViewerRequestS  ervices  S  ervice  .wsdl 

DoDConnector.wsdl 

CHSConnector.wsdl 

WAR  Files 

UniversalClientGUI.war 

UniversalClientGUI.war 

UniversalClientWS.war 

UniversalClientW  S  .war 

XML  Files 

adapterServicesMappings.xml 

adapterServicesMappings.xml 

XSL  Files 

CCD.XS1 

CCDCHS.xsl 

Allscripts 

Folders 

allscriptsdataset 

soapextender 

webservices 

Table  34.  MIDHT  Code  Delivered  to  TATRC/DHIMS/FHA. 
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CONCLUSION 


MIDHT  implemented  and  analyzed  various  health  information  technologies  throughout 
Conemaugh  Health  System  for  the  respective  contract.  Technologies  were  envisioned  to  assist 
both  providers  and  patients  in  the  journey  to  improve  care  coordination,  workflow,  duplicate 
testing  and  rising  costs.  The  final  results  were  mixed  as  major  new  investments  take  time  to 
mature  and  the  impact  on  people  must  be  well  planned  and  communicated  effectively. 

Our  experience  demonstrated  that  EHR  systems  produce  many  benefits  to  users  but  may  also 
create  new  problems.  Moving  from  a  primarily  paper-based  record  system  to  an  electronic 
environment  is  complex  and  there  will  be  bumps  along  the  way.  hnplementers  must  not  only 
focus  on  the  technical  product  itself  but  also  must  plan  accordingly  during  transition  times  and 
determine  how  work  redesign  will  impact  staff  It  will  be  of  great  importance  to  set  staff 
expectations  in  advance  and  keep  communication  lines  open  during  the  implementation  process. 

As  providers  implement  EHR  systems  spurred  by  meaniungful  use  stimulus  funding  and  thus 
have  clinical  data  residing  in  electronic  form,  interoperability  between  systems  becomes 
achievable  and  critically  important  in  improving  continuity  of  care  and  reducing  costs.  MIDHT 
provided  valuable  technical  resources  to  assist  the  Department  of  Defense  in  achieving 
milestones  for  the  Virtual  Eifetime  Electronic  Record  (VEER)  la  project  in  San  Diego, 
California.  Furthermore,  valuable  experience  was  gained  by  Conemaugh  and  Northrop 
Grumman  staff  with  the  CONNECT  software,  CDA  specifications,  CAL  architecture,  and  C32 
dynamic  document  generation  which  laid  the  framework  for  Conemaugh  to  become  the  first  non¬ 
governmental  institution  in  the  state  of  Pennsylvania  to  become  a  production  member  of  the 
Nationwide  Health  Information  Network  during  a  subsequent  MIDHT  award. 

The  implementation  of  PACS  technology  allows  providers  in  disparate  geographic  locations  to 
access  images  (for  multiple  modalities)  performed  off-site  and  encourages  consultation  between 
referring  and  receiving  institutions.  The  implementation  of  PACS  is  especially  valuable  in  a 
rural  healthcare  system  such  as  Conemaugh  consisting  of  an  urban  flagship  medical  center  and 
two  small  hospitals.  The  reduction  in  unnecessary  duplicate  testing  will  continue  to  increase  as 
more  physicians  embrace  said  technology  and  begin  to  change  ordering  habits. 

PHR’s  have  generated  minimal  interest  and  usage  by  subjects  and  providers  to  date.  Our 
findings  align  with  patient  attitudes  nationwide.  Adoption  of  said  technology  remains  below 
10%  and  significant  usage  can  only  be  found  in  geographic  pockets  or  within  specific  patient 
populations.  As  the  nation  is  currently  focused  on  “meaningful  use”  of  EHR  systems,  wide¬ 
spread  adoption  of  PHRs  may  be  five  years  away  from  reality. 
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Appendix  1  -  Electronic  Health  Record  (EHR)  Implementation  Survey 


1.  The  instructions  and  prompts  are  helpful. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

12.0% 

3 

Sometimes 

52.0% 

13 

Occasionally 

4.0% 

1 

Most  of  the  time 

28.0% 

7 

Always 

4.0% 

1 

answered  question 

25 

skipped  question 

1 

2.  Getting  paper-based  documents  in  and  out  of  the  system  is  easy. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

3.8% 

1 

Sometimes 

19.2% 

5 

Occasionally 

19.2% 

5 

Most  of  the  time 

50.0% 

13 

Always 

7.7% 

2 

answered  question 

26 

skipped  question 

0 

3.  1  sometimes  wonder  if  I'm  using  the  right  command. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

15.4% 

4 

Sometimes 

30.8% 

8 

Occasionally 

34.6% 

9 

Most  of  the  time 

19.2% 

5 

Always 

0.0% 

0 

answered  question 

26 

skipped  question 

0 

4.  The  speed  of  the  EHR  is  fast  enough  to  accomplish  tasks. 

Answer  Options 

Response 

Percent 

Response 

Count 

Agree 

42.3% 

11 

Disagree 

53.8% 

14 

Not  Sure 

3.8% 

1 

answered  question 

26 

skipped  question 

0 
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5.  This  software  seems  to  disrupt  the  way  1  normally  like  to  arrange  my  work. 

Answer  Options 

Response 

Percent 

Response 

Count 

Agree 

50.0% 

13 

Disagree 

38.5% 

10 

Not  Sure 

11.5% 

3 

answered  question 

26 

skipped  question 

0 

6.  The  organization  of  the  menus  or  information  lists  seems  quite  logical. 
Answer  Options 


Agree  50.0%  13 

Disagree  50.0%  13 

Not  Sure  0.0%  0 

answered  question  26 

skipped  question  0 


7.  It  is  relatively  easy  to  move  from  one  part  of  a  task  to  another. 

Answer  Options 

Response 

Percent 

Response 

Count 

Agree 

34.6% 

9 

Disagree 

57.7% 

15 

Not  Sure 

7.7% 

2 

answered  question 

26 

skipped  question 

0 

8.  It  is  easy  to  forget  how  to  do  things  with  the  EHR. 

Answer  Options 

Response 

Percent 

Response 

Count 

Agree 

34.6% 

9 

Disagree 

53.8% 

14 

Not  Sure 

11.5% 

3 

answered  question 

26 

skipped  question 

0 

9.  The  EHR  occasionally  performs  in  a  way  which  can't  be  logically  understood. 

Answer  Options 

Response 

Percent 

Response 

Count 

True 

69.2% 

18 

False 

30.8% 

8 

answered  question 

26 

skipped  question 

0 
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10.  I  have  to  seek  assistance  most  times  when  I  use  the  EHR. 


Answer  Options 

True 

False 


Response 

Percent 

16.0% 

84.0% 

answered  question 
skipped  question 


Response 

Count 


4 

21 


25 

1 


1 1 .  How  often  can  you  count  on  the  EHR  to  be  up  and  available. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

0.0% 

0 

Sometimes 

3.8% 

1 

Occasionally 

3.8% 

1 

Most  of  the  time 

73.1% 

19 

Always 

19.2% 

5 

answered  question 

26 

skipped  question 

0 

12.  How  often  is  the  EHR  subject  to  frequent  problems  and  crashes. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

7.7% 

2 

Sometimes 

23.1% 

6 

Occasionally 

69.2% 

18 

Most  of  the  time 

0.0% 

0 

Always 

0.0% 

0 

answered  question 

26 

skipped  question 

0 

13.  The  EHR  provides  me  with  all  the  information  1  need  to  take  care  of  the  patient. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

0.0% 

0 

Sometimes 

23.1% 

6 

Occasionally 

7.7% 

2 

Most  of  the  time 

61.5% 

16 

Always 

7.7% 

2 

answered  question 

26 

skipped  question 

0 
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14.  The  EHR  screens  include  a  lot  of  extra  information  that  I  don’t  need. 

Answer  Options 

Response 

Percent 

Response 

Count 

Agree 

19.2% 

5 

Disagree 

65.4% 

17 

Not  Sure 

15.4% 

4 

answered  question 

26 

skipped  question 

0 

15.  There  is  inaccurate  information  in  the  EHR. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

16.0% 

4 

Sometimes 

64.0% 

16 

Occasionally 

16.0% 

4 

Most  of  the  time 

4.0% 

1 

Always 

0.0% 

0 

answered  question 

25 

skipped  question 

1 

16.  The  EHR  provides  information  that  is  up-to-date. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

0.0% 

0 

Sometimes 

11.5% 

3 

Occasionally 

3.8% 

1 

Most  of  the  time 

76.9% 

20 

Always 

7.7% 

2 

answered  question 

26 

skipped  question 

0 

17.  The  system  lets  me  quickly  find  the  information  1  need. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

0.0% 

0 

Sometimes 

40.0% 

10 

Occasionally 

28.0% 

7 

Most  of  the  time 

24.0% 

6 

Always 

8.0% 

2 

answered  question 

25 

skipped  question 

1 
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18.  The  information  in  the  EHR  is  presented  in  a  useful  format. 

Answer  Options 

Response 

Percent 

Response 

Count 

Yes 

30.8% 

8 

No 

15.4% 

4 

Depends  on  specific  functionaiity 

53.8% 

14 

answered  question 

26 

skipped  question 

0 

19.  The  information  in  the  system  includes  the  level  of  detail  that  1  need. 

Answer  Options 

Response 

Percent 

Response 

Count 

Yes 

42.3% 

11 

No 

15.4% 

4 

Depends  on  specific  functionaiity 

42.3% 

11 

answered  question 

26 

skipped  question 

0 
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20.  Compared  to  previous  routines,  how  has  the  EHR  changed  the  performance  of  the  following 

tasks? 

Answer  Options 

Much  More 
Difficult 

Slightly  More 
Difficult 

No 

Change 

Slightly 

Easier 

Much 

Easier 

N/A,  Don’t 
Know 

Response 

Count 

Documenting  physical  exams 

3 

6 

1 

1 

4 

11 

26 

Documenting  histories 

4 

5 

1 

1 

5 

10 

26 

Documenting  allergies 

1 

2 

4 

2 

7 

9 

25 

Documenting  CPT  and  ICD-9  codes  for  billing  purposes 

2 

0 

3 

3 

4 

14 

26 

Keeping  problem  lists  updated 

2 

3 

2 

6 

6 

7 

26 

Keeping  medication  lists  updated 

3 

4 

3 

2 

6 

8 

26 

Ordering  laboratory  and  radiology  tests 

6 

2 

1 

2 

3 

12 

26 

Reviewing  laboratory  and  radiology  results 

2 

2 

3 

5 

7 

7 

26 

Writing  prescriptions 

2 

3 

1 

3 

7 

10 

26 

Renewing  prescriptions 

2 

1 

3 

3 

9 

8 

26 

Monitoring  medication  safety  during  prescribing 

1 

1 

1 

5 

4 

14 

26 

Monitoring  patient  medication  adherence 

1 

0 

4 

2 

3 

16 

26 

Communicating  referral  information  to  specialists 

2 

0 

4 

7 

2 

11 

26 

Reviewing  referral  information  from  specialists 

1 

1 

9 

5 

2 

8 

26 

Ordering  appropriate  preventive  care  services 

1 

4 

3 

3 

2 

13 

26 

Making  a  list  of  patients  based  on  diagnosis  or  history 

1 

0 

0 

3 

1 

21 

26 

Contacting  patients  to  remind  them  of  appointments 

0 

0 

4 

2 

2 

18 

26 

Assisting  patients  in  self-management  activities 

1 

1 

2 

2 

2 

18 

26 

answered  question 

26 
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21 .  How  strongly  do  you  agree  or  disagree  with  the  following  statements  regarding  the  EHR? 


Answer  Options 

Completely 

Generally 

Generally 

Completely 

Don’t 

N/A 

Response 

Agree 

Agree 

Disagree 

Disagree 

Know 

Count 

Using  the  EHR  has  enabled  me  to  accomplish  tasks 
quicker 

2 

11 

9 

4 

0 

0 

26 

I  work  longer  hours  to  see  the  same  number  of  patients 

8 

4 

4 

4 

2 

4 

26 

Using  the  EHR  has  enhanced  my  effectiveness  in  my  job 

2 

12 

8 

4 

0 

0 

26 

Using  the  EHR  has  made  it  easier  to  do  my  job 

3 

11 

8 

4 

0 

0 

26 

I  find  the  EHR  useful  in  my  job 

2 

16 

5 

2 

1 

0 

26 

Learning  to  operate  the  EHR  has  been  easy  for  me 

1 

15 

8 

2 

0 

0 

26 

I  have  become  skilled  at  using  the  advanced  features 

2 

12 

5 

0 

4 

3 

26 

Easier  to  access  patient  information  from  outside  the 
office 

2 

5 

1 

0 

3 

15 

26 

There  are  too  many  alerts  and  reminders 

3 

7 

3 

1 

4 

8 

26 

Has  decreased  the  amount  of  time  I  spend  talking  to 
patients 

4 

6 

7 

2 

3 

4 

26 

Helps  me  adhere  to  clinical  practice  guidelines 

1 

7 

2 

0 

6 

10 

26 

Using  an  EHR  has  caused  disruptions  to  my  work  flow 

4 

8 

8 

3 

1 

2 

26 

Has  improved  my  ability  to  make  decisions  about  patient 

1 

6 

4 

1 

3 

10 

25 

care 

Has  improved  my  ability  to  provide  preventive  care 

1 

6 

3 

2 

4 

10 

26 

I  can  better  monitor  how  many  of  my  patients  are  receiving 
appropriate  care 

1 

4 

5 

1 

2 

13 

26 

Benefits  of  adopting  an  EHR  have  outweighed  the 
challenges 

2 

10 

5 

3 

4 

2 

26 

answered  question 


26 
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22.  Would  you  recommend  EHRs  to  other  providers  interested  in  adopting  health 

information  technology? 

Answer  Options 

Response 

Percent 

Response 

Count 

Yes 

72.0% 

18 

No 

28.0% 

7 

answered  question 

25 

skipped  question 

1 

23.  If  you  could  change  one  thing  about  the  Allscripts  EHR  system, 

what  would  it  be? 

Answer  Options 

Response 

Count 

18 

answered  question 

18 

skipped  question 

8 

24.  What  has  been  the  most  positive  benefit  of  using  an  EHR? 

Answer  Options 

Response 

Count 

18 

answered  question 

18 

skipped  question 

8 

25.  How  often  do  you  print  out  a  Visit  Summary  report  from  the  EHR  system  for 
your  patient  at  the  conciusion  of  the  patient  visit? 


Answer  Options 

Response 

Percent 

Response 

Count 

Always 

0.0% 

0 

Sometimes 

20.0% 

5 

Never 

44.0% 

11 

Don’t  know 

12.0% 

3 

Not  Applicable 

24.0% 

6 

answered  question 

25 

skipped  question 

1 

26.  How  long  have  you  been  using  the  Allscripts  EHR  at  your 

practice? 

Answer  Options 

Response 

Count 

26 

answered  question 

26 

skipped  question 

0 
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27.  How  would  you  classify  your  level  of  comfort  with  general  computer  technology 

(e.g.  email,  Internet,  word  processing)? 

Answer  Options 

Response 

Percent 

Response 

Count 

Very  comfortable 

53.8% 

14 

Somewhat  comfortable 

46.2% 

12 

Not  very  comfortable 

0.0% 

0 

answered  question 

26 

skipped  question 

0 

28.  Please  choose  the  statement  that  best  describes  the  training  you  received  for 

your  current  EHR. 

Answer  Options 

Response 

Percent 

Response 

Count 

1  have  received  no  training  and  I’m  learning  the 
system  as  1  use  it 

0.0% 

0 

Informal  training  by  practice  staff  when  time 
permitted 

3.8% 

1 

Less  than  ten  hours  dedicated  to  formal  training 
(with  vendor  or  practice  trainers) 

53.8% 

14 

Ten  or  more  hours  dedicated  to  formal  training  (with 

11 

vendor  or  practice  trainers) 

answered  question 

26 

skipped  question 

0 

29.  What  is  your  title? 

Answer  Options 

Response 

Percent 

Response 

Count 

Clerical 

37.5% 

9 

Nurse 

33.3% 

8 

Office  Manager 

4.2% 

1 

Physician 

8.3% 

2 

Physician  Assistant 

0.0% 

0 

Other 

16.7% 

4 

answered  question 

24 

skipped  question 

2 

30.  Do  you  have  any  previous  EHR  experience? 


Answer  Options 

Yes 

No 


Response 

Percent 

26.9% 

73.1% 

answered  question 
skipped  question 


Response 

Count 


7 

19 


26 

0 
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31 .  How  satisfied  are  you  with  the  EHR  system? 

Answer  Options 

Response 

Percent 

Response 

Count 

Very  Dissatisfied 

11.5% 

3 

Dissatisfied 

23.1% 

6 

Neutral 

30.8% 

8 

Somewhat  Satisfied 

30.8% 

8 

Very  Satisfied 

3.8% 

1 

answered  question 

26 

skipped  question 

0 

32.  Please  provide  additional  comments  as  needed. 

Answer  Options 

Response 

Count 

answered  question 

6 

6 

skipped  question 

20 
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Appendix  2  -  Protocol  Closure  Memorandum 


Page  I  of 


John  Hargreaves 


From:  BnQBch.  Lbumb  R  Dr  CIV  USA  MEDCQM  USAMRFilG  [Lmjra.BrcKBchi^us.armi^.niil] 
sent:  Thiraday,  April  19, 201 2  2:1  &  PM 

TO;  Brian  Lieb 
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Abbreviated  Title  Page 


a)  The  Impact  of  System-Wide  Image  Access  on  Duplicate  Testing  in  a  Rural  Healthcare 
System. 

b)  Original  Research 

c)  Advance  in  Knowledge 

1 .  The  ability  for  the  receiving  institution  to  view  images  taken  off-site  through  a 
picture  archiving  and  communications  system  (PACS)  resulted  in  a  7%  reduction 
in  duplicate  chest  x-ray  and  CT  scan  (head)  testing  from  0-7  days  when  compared 
to  a  pre  PACS  implementation  period. 

2.  As  surveyed,  seventy  percent  of  physicians  positively  stated  that  immediate  image 
access  did  reduce  the  number  of  exams  reordered. 

d)  Implications  for  Patient  Care 

1 .  System-wide  availability  of  images  will  have  a  positive  impact  and  may  reduce 
unnecessary  imaging  at  the  receiving  institution.  Benefits  will  also  transfer  to 
patients  as  health  risks  associated  with  radiation  exposure  will  decrease. 

2.  Implementers  should  have  a  solid  plan  in  place  to  communicate  and  train  users  on 
the  new  functionality.  Leadership  should  expect  that  change  management  will 
vary  amongst  physicians  and  may  impact  immediate  results. 

e)  Summary  Statement 

The  implementation  of  picture  archiving  and  communications  system  (PACS)  throughout  a 
rural  health  care  system  will  have  positive  benefits  both  to  patients  and  the  institutions 
involved  and  likely  reduce  unnecessary  imaging,  operating  costs  and  radiation  exposure  to 
patients. 
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Abstract 


Purpose: 

To  test  the  hypothesis  that  duplicate  imaging  will  decrease  among  transfer  patients  to  a  tertiary 
care  center  from  two  rural  off-site  hospitals  after  the  implementation  of  picture  archive  and 
communications  system  (PACS)  at  the  off-sites. 

Materials  and  Methods: 

This  minimal  risk,  HIPAA-compliant  study  was  approved  by  Conemaugh’s  Institutional  Review 
Board  (IRB),  with  waiver  for  informed  consent  for  retrospective  review  of  medical  records. 

Using  a  master  patient  index  (MPI),  625  duplicate  chest  x-rays  and  CT  scans  of  the  head  between 
sending  and  receiving  institution  (taken  within  0-7  days)  were  collected  from  July  2008  to  June 
2010.  The  study  design  utilized  a  pre  vs.  post  quantitative  methodology,  with  time  periods  based 
upon  the  extension  of  PACS  technology  to  off-site  locations.  Additionally,  qualitative  feedback 
was  gathered  from  physicians  (n=76)  using  a  survey  tool  to  assess  the  impact  of  immediate  image 
access  and  to  quantify  the  number  of  off-site  studies  viewed  by  physicians  (n=70)  with  PACS 
access. 

Results: 

A  Chi-Square  test  of  independence  applied  to  either  Days  between  date  of  service  or  Aggregated 
Days  between  date  of  service  over  time  period  did  not  yield  a  statistically  significant  result 
despite  a  7%  reduction  in  duplicate  imaging  (hypothesis  not  accepted).  A  financial  analysis  of 
the  resulting  seven  percent  reduction  in  duplicate  tests  suggests  a  savings  of  $187,075  to  patients 
and/or  insurance  companies. 

Conclusion: 

Extension  of  PACS  technology  to  referring  institutions  is  beneficial;  however,  realization  of  a 
significant  reduction  in  duplicate  testing  will  depend  upon  full  support  of  ordering  physicians, 
proper  training,  and  effective  communication. 
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Introduction 


The  United  States  spends  more  on  health  care  than  any  other  country  with  an  annual  average  of 
$6,401  per  person,  which  is  2.4  times  the  average  of  developed  countries  (1).  As  hospital 
reimbursement  becomes  more  challenging,  health  information  technology  (HIT)  may  offer 
solutions  to  achieve  organization-wide  cost  savings.  Duplicate  testing  is  not  only  a  well-known 
source  of  extraneous  health  care  expenses  but  may  also  pose  additional  radiation  risks  to  patients. 
As  a  rural  health  care  system  with  multiple  referring  hospitals,  how  can  efficiency  and  patient 
safety  improve  through  coordinated  care? 

Duplicate  testing  remains  an  industry-wide  challenge  that  must  be  addressed  in  order  for  health 
care  reform  to  be  realized.  Haley  et  al  found  that  53%  of  transferred  trauma  patients  had  some 
portion  of  their  images  duplicated;  resulting  in  $650,000  in  additional  costs  (2).  Thomas  et  al 
presented  similar  findings  in  that  43%  of  patients  had  computed  topography  (CT)  scans  repeated 
during  facility  transfers  (3). 

Recent  studies  have  stressed  the  inherent  risk  to  patients  due  to  increased  source  of  radiation 
exposure.  Brenner  and  Hall  state  that  62  million  CT  scans  are  performed  annually  in  the  United 
States  and  involve  larger  radiation  doses  than  more  conventional  x-ray  tests  (4).  Berrington  de 
Gonzalez  et  al  also  provide  support  for  increased  cancer  risk  estimating  that  29,000  future 
cancers  could  be  related  to  CT  scans  (5).  Sodickson  et  al  recommend  the  quantity  of  imaging 
should  be  monitored  over  time  to  ensure  minimization  of  radiation  exposure  (6). 
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One  solution  that  may  reduce  unnecessary  radiology  testing  and  reduce  radiation  exposure 
between  central  and  remote  sites  is  called  Picture  Archiving  and  Communication  System 
(PACS).  PACS  are  computer  networks  dedicated  to  the  storage,  retrieval,  and  presentation  of 
images  produced  by  medical  imaging  devices.  PACS  replaces  film  archives,  allowing  imaging 
access  simultaneously  and  from  off-site  locations.  PACS  is  commonly  believed  to  reduce  the 
number  of  unnecessary  duplicate  imaging  tests  ordered  because  of  originals  being  lost  or  stored 
at  a  remote  location.  Past  research  suggests  an  individualized  evaluation  of  PACS  technology 
where  incidence  of  duplicate  testing  may  be  high  (7).  Institutions  involved  with  the  transfer  of 
patients  may  have  the  most  to  gain  through  this  process. 

Materials  &  Methods 

This  study,  located  within  the  Conemaugh  Health  System  (CHS),  analyzed  the  impact  of 
extending  PACS  to  two  remote  hospitals  stretched  over  a  two  county  area  in  southwestern 
Pennsylvania.  Conemaugh  Memorial  Medical  Center  (central  site),  located  in  Johnstown, 
Pennsylvania,  is  a  tertiary  care  regional  referral  hospital  known  for  clinical  excellence  and  patient 
satisfaction.  The  level  1  trauma  center  located  at  the  central  site  is  one  of  just  eleven  centers  in 
Pennsylvania.  Miners  Medical  Center  (remote  site  1)  is  a  30-bed  community  satellite  hospital 
located  45  minutes  to  the  north  of  Johnstown  whereas  Meyersdale  Medical  Center  (remote  site  2) 
is  a  20-bed  Critical  Care  Access  hospital  located  60  minutes  to  the  south  of  Johnstown.  The 
study  was  approved  by  Conemaugh’s  Institutional  Review  Board  (IRB),  which  included  a  waiver 
of  informed  consent  for  retrospective  review  of  medical  records  and  a  minimal  risk  designation. 
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Implementation 

Conemaugh  extended  the  MeKesson  PACS,  Radiology  Information  System  (RIS)  and  Dolby 
Digital  Dictation  system  used  at  the  central  site  to  the  two  remote  facilities  (Figure  1).  This 
project  allowed  CHS  to  achieve  consistency  of  radiology  imaging,  report  management,  and 
image  access  across  the  health  system.  PACS  went  live  at  the  remote  sites  on  July  1,  2009  (image 
transfer  only).  During  the  next  six  months,  activities  were  completed  to  seamlessly  integrate 
both  the  RIS  and  PACS  systems  for  sharing  of  reports  in  production  use. 


Figure  1.  Chest  x-ray  within  PACS. 

Study  Design 

Both  qualitative  and  quantitative  methods  were  employed  to  investigate  the  study  objectives  and 
the  hypothesis.  Researchers  hypothesized  that  the  number  of  duplicated  chest  X-rays  and  CT 
scans  (head)  would  significantly  decrease  after  implementation  of  each  phase  of  the  PACS 
implementation  compared  to  that  of  baseline. 
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Data  collection  primarily  utilized  historical  data  (retrospectively)  to  obtain  the  number  of 
duplicated  diagnostic  imaging  tests  by  evaluating  empirical  data  retrieved  from  hospital  financial 
systems.  The  data  was  limited  to  patients  that  were  first  treated  at  one  of  the  two  remote  sites  and 
subsequently  treated  at  the  central  site  within  the  defined  duplicate  test  time  frame  (0-7  days). 
This  comparative  analysis  used  a  PRE  (before  PACS)  and  POST  (after  PACS)  time  period,  see 
Table  I,  design  to  ascertain  the  expected  change  in  the  viewing  of  PACS  images.  The  time 
periods  were  chosen  with  consideration  for  symmetry  of  data  sets,  stabilization  of 
implementation,  and  the  possibility  of  seasonality  (time)  as  a  covariate. 


PRE 

(baseline) 

P  o 

S  T 

Phase  1 

I  Phase  2 

2008 

2009 

'  2010 

July  through 

January  through 

July  through 

I 

'  January  through 

December 

June 

December 

June 

le  I.  Study  Timeline. 

Conemaugh  MIS  department  provided  duplicate  test  reports  to  the  study  team  for  analysis  using 
the  Master  Patient  Index  (MPI)  software.  The  report  included  chest  x-rays  (CPT  71010  and 
71020)  and  CT  scans  of  the  head  (CPT  70450)  for  the  stated  time  periods.  The  report  included 
data  for  patients  that  had  the  same  test,  as  specified  by  the  CPT  codes  above,  and  determined  by 
the  patient  identifier,  date  of  service,  and  service  location.  Inclusion  of  data  was  limited  to 
studies  repeated  at  the  central  site  within  0-7  days.  Time  of  service  was  used  to  include/exclude 
tests  performed  on  the  same  date.  Additional  images  completed  at  CMMC  were  removed  from 
the  analysis.  Summary  radiology  volume  data  was  also  collected  to  assess  the  level  of 
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consistency  of  volume  over  time  and  thereby  providing  one  measure  of  the  homogeneity  of  the 
pre  and  post  data  sets. 

In  order  to  understand  actual  CMMC  physician  usage  of  studies  performed  at  the  two  rural 
hospitals,  PACS  User  Reports  were  provided  to  the  study  team  for  analysis.  The  population 
included  a  randomized,  proportionate  sample  of  active  physicians  on  the  CMMC  medical  staff 
Reports  were  prepared  for  October  -  December  2009  and  April  -  June  2010,  which  detailed 
viewing  of  studies  that  originated  from  the  remote  sites. 

A  physician  opinion  survey  was  also  distributed  to  all  applicable  physicians  on  the  CMMC 
medical  staff  in  traditional  hard  copy  and  online  form.  The  survey  was  designed  by  Canada 
Health  Infoway  and  was  modified  for  use  by  Conemaugh  with  permission.^  The  survey  was 
designed  to  gather  qualitative  feedback  from  physicians  that  use  the  system  daily  in  their  course 
of  patient  care.  Survey  objectives  included  assessing  the  impact  of  immediate  image  access  to 
studies  performed  at  two  rural  hospitals  on  productivity,  decision-making,  patient  transfers,  and 
duplicated  exams. 

Results 

A  Chi-Square  test  of  independence  applied  to  either  Days  between  (b/w)  date  of  service  (DOS) 
or  Aggregated  Days  b/w  DOS  over  time  period  does  not  yield  a  statistically  significant  result 
despite  a  7%  reduction  in  duplicate  imaging  (hypothesis  not  accepted).  However,  a  Chi-Square 
test  of  goodness-of-fit  for  aggregated  Days  b/w  DOS  using  pre  data  as  the  expected  (population) 
values  and  the  post  data  as  observed  values  does  yield  a  statistically  significant  result  at  a  family- 
wise  alpha  =  0.05.  This  result  implies  that  the  implementation  of  the  PACS  system  at  the  remote 
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sites  did  contribute  to  the  change  in  the  distribution  of  the  count  of  duplicative  testing  (Figure  2). 
Furthermore,  no  statistical  significance  was  found  by  CPT  code  or  location  (Figures  3-4). 
Radiology  volume  data  by  CPT  code  for  the  remote  sites  is  homogeneous  between  the  pre  and 
post  data  sets. 


Days  b/w  DOS,  aggregated 


Figure  2.  Duplicated  Tests  (PRE  vs.  POST)  by  Days  Between  Date  of  Service. 
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CT  scan,  head  Chest  X-ray  Chest  X-ray 

(single  riew)  (double  fieir) 


Test  (Image)  Tj^ie 
Figure  3.  Duplicated  Tests  (PRE  vs.  POST)  by  CPT  Code. 


Time  Period 

Ipre 

■post 


Remote  Site  2  Remote  Site  1 

HOSPITAL_OFF-site 

Figure  4.  Duplicated  Tests  (PRE  vs.  POST)  by  Initial  Location. 
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Despite  non-significant  findings  in  the  reduction  of  duplicate  testing,  the  financial  analysis  of  the 
resulting  seven  percent  reduction  in  duplicated  chest  x-rays  and  CT  scans  of  the  head  suggests  a 
savings  of  $187,075  to  patients  and/or  insurance  companies.  As  healthcare  costs  continue  to 
climb  and  funding  becomes  more  restricted,  hospitals  and  health  systems  must  consider 
implementing  health  information  technologies  to  improve  efficiencies  and  save  money.  Though 
not  specifically  addressed  in  this  study,  the  reduction  in  duplicate  testing  will  translate  into  less 
radiation  exposure  opportunities  for  patients  transferred  between  facilities. 

The  following  data  was  collected  for  a  randomized,  proportionate  sample  of  70  physicians  with 
PACS  access.  As  depicted  below  in  Figure  5,  the  most  active  users  of  the  PACS  system  in  terms 
of  viewing  studies  originating  from  MIMC  and  MYMC  are  Emergency  Medicine  (n=65)  and 
Trauma  (n=51)  physicians.  This  result  is  expected  as  CMMC  is  a  tertiary  care  referral  hospital 
with  a  Level  1  trauma  center.  The  next  grouping  includes  Otolaryngology  (n=35).  Urology 
(n=34),  and  Pulmonary  (n=31).  A  third  grouping  includes  General  Surgery  (n=28).  Orthopedics 
(n=27)  and  Neurosurgery  (n=26).  The  graph  does  not  include  an  outlier  for  an  orthopedic 
surgeon  (n=581). 
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Figure  5.  Number  of  studies  performed  off-site  viewed  using  PACS. 

Qualitative  feedback  from  system  users  was  collected  via  a  survey  tool.  Seventy  six  physicians 
completed  the  survey  during  October  2010  through  January  2011,  representing  approximately  a 
37%  response  rate.  The  following  graph  depicts  self  reported  frequency  of  electronic  access 
(Frequently,  Sometimes,  Seldom,  or  Never)  of  images  originating  from  MIMC  and  MYMC  by 
stated  specialty.  Results  do  align  with  empirical  usage  data  for  highest  volume  specialties: 
General  Surgery,  Trauma,  Emergency  Medicine,  Internal  Medicine,  Orthopedics,  Urology  (not 
shown).  Neurosurgery,  Otolaryngology  (not  shown),  and  Pulmonary  (not  shown).  Family 
practitioners  responded  the  most  but  actual  usage  of  the  system  is  limited. 
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Figure  6.  Survey  Respondents  by  Specialty  and  Frequency  of  PACS  Usage. 


Surveys  that  indicated  that  the  physician  never  used  PACS  to  access  images  from  the  remote 
sites  were  removed  from  the  dataset  before  analysis.  The  remaining  responses  (n=55)  formed  the 
dataset  of  analysis. 


The  most  important  survey  question  in  terms  of  the  hypothesis  was  Question  10a:  Immediate 
PACS  image  access  to  films  performed  at  Miners/Meyersdale  has  reduced  the  number  of  exams 
reordered  because  the  images  were  not  available  when  I  needed  them?  Seventy  percent  (70%)  of 
physicians  that  use  the  PACS  system  for  this  purpose  either  Strongly  Agreed  or  Moderately 
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Agreed  that  the  number  of  duplicated  tests  has  been  reduced  post  implementation  whereas  the 


remaining  30%  disagreed  or  stated  it  was  not  applicable. 


Physician  Opinion 

Figure  7.  Self-reported  Physician  Opinion  on  Duplicate  Testing  by  Frequency  of  PACS  Usage. 


Hypothesis  testing  of  this  question  and  others  indicate  that  the  response  per  category  most 
probably  represents  a  real  difference  in  respondent  opinion.  The  results  of  the  hypothesis  testing 
do  not  change  for  the  aggregations  of  the  responses  of  those  questions.  [Asymptotic  significance 
at  the  0.05  level  is  shown  for  all  hypothesis  testing.]  Questions  were  analyzed  using  the  original 
scale,  a  5  level  Likert  scale,  and  an  aggregation  (collapsing)  of  that  scale  comprised  of  3  levels 
(Agree,  Disagree,  N/A). 
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Discussion 


The  implementation  of  PACS  teehnology  at  smaller  hospitals  within  the  same  health  system  is 
benefieial  and  likely  to  reduee  costs  and  radiation  exposure  to  patients.  This  study  found  that 
positive  qualitative  feedback  and  modest  system  usage  by  physicians  did  translate  into  a 
reduction  in  duplicate  testing  although  not  significant  for  patients  receiving  care  from  multiple 
facilities  within  the  same  rural  health  system.  Despite  new  technologies  being  available, 
physicians  must  take  the  appropriate  amount  of  time  to  receive  education  and  attend  training 
sessions.  Unfortunately,  some  physicians  may  not  change  their  ordering  habits,  which  will 
reduce  the  expected  benefit  significantly  depending  on  position  and  corresponding  volume. 

One  limitation  of  the  study  is  that  the  reason  for  ordering  the  test  was  not  consistently  collected 
and  therefore  unavailable  for  analysis  by  investigators.  Such  analysis  would  have  been  beneficial 
to  determine  how  many  more  duplicate  tests  could  have  been  prevented  if  the  physician  had  been 
aware  of  the  previous  test  conducted  off-site  or  the  test  was  re-ordered  based  upon  sound  clinical 
judgment. 

To  enhance  data  analysis  and  study  conclusions,  future  research  should  attempt  to  collect 
reasoning  information  used  in  deciding  to  reorder  an  image.  Given  the  surge  in  new  technology 
investments  nationwide  from  American  Recovery  and  Reinvestment  Act  (ARRA)  funding,  future 
studies  should  be  conducted  on  health  information  exchanges  (HIE)  that  share  images  across  the 
street  or  across  state  lines.  An  understanding  of  the  cost  benefit  analysis  of  said  HIE  projects  will 
be  important  to  move  the  industry  forward. 
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Appendix  5  -  MIDHT  Clinical  Viewer  -  Patient  Search 


"V  Search  for  a  Patieitt  -  MIDHT  Health  Informatjo-n  Exchange  -  Microsoft  Internet  Explorer  providecf  by  Conentaugh  Healtit  System, 
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Research  Acquisition  Activity,  S20  Chandler  Street,  Fort  Detrick  MD  21702-5014  is  the  awarding  and  administering  acquisition 
oftice.  This  information  does  not  necessarily  reflect  the  position  or  the  policy  of  the  Government,  and  no  official  endorsement 
should  be  inferred. 
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MIDHT  Clinical  Viewer  -  Patient  found  utilizing  Initiate  EMPI  web  service 


^  Patient  Search  Results  -  MIDHT  Health  Informatjon  Exchange  -  Microsoft  Internet  Explorer  provided  by  Conemaugh  Health  System 
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office.  This  information  does  not  necessarily  reflect  the  p-osition  or  the  policy  of  the  Government,  and  no  official  endorsement 
should  be  inferred. 
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Conemaugh  provider  retrieving  DoD  (AHLTA)  data  from  “Document  Inbox” 
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DoD  provider  searching  for  patient  -  AHLTA  Client  (test) 
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DoD  provider  retrieving  Conemaugh  data  from  “NHiN  Documents”  folder 
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Appendix  6  -  Health  Information  Exchange  Architecture 


http://www.hitsp.org/ConstructSet  Details.aspx?&PrefixAlpha=4&PrefixNumeric=32 
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